2024-09-16T08:05:56.791 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T08:05:56.807 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T08:05:56.885 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906926 branch: wip-guits-main-2024-09-13-1248 description: orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream-reef 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} email: gabrioux@ibm.com first_in_suite: false job_id: '7906926' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-13-1248 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - CEPHADM_AGENT_DOWN log-only-match: - CEPHADM_ sha1: 8293d73f8690540e843a81caec373f9cc29cf705 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 8293d73f8690540e843a81caec373f9cc29cf705 workunit: branch: wip-guits-main-2024-09-13-1248 sha1: 8293d73f8690540e843a81caec373f9cc29cf705 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b seed: 6714 sha1: 8293d73f8690540e843a81caec373f9cc29cf705 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-13-1248 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8293d73f8690540e843a81caec373f9cc29cf705 targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKJBHN4PORr3u0gCJNBcZXYFQVc4sQCOyHsxGRxuz2oaRxWLuS1R6AUDoyCAPwOZ3L/8anKbT5v4e1pHMKBW3RQ= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFI3q8MO/o/QqKIi0uyIf+VkDGR6NWb3qnObNsRWyMCIMfoZyuXgySPpDqcLxoDSQ1VMVI2Yp5ovbSPSjoyC2oM= tasks: - cephadm: compiled_cephadm_branch: reef image: quay.ceph.io/ceph-ci/ceph:reef - cephadm.shell: env: - sha1 mon.a: - radosgw-admin realm create --rgw-realm=r --default - radosgw-admin zonegroup create --rgw-zonegroup=default --master --default - radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default - radosgw-admin period update --rgw-realm=r --commit - ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000 - ceph orch apply rgw smpl - ceph osd pool create foo - rbd pool init foo - ceph orch apply iscsi foo u p - sleep 120 - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 mon.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph orch upgrade status - ceph health detail - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 - ceph orch ls | grep '^osd ' - cephadm.shell: mon.a: - ceph orch upgrade ls - ceph orch upgrade ls --image quay.io/ceph/ceph --show-all-versions | grep 16.2.0 - ceph orch upgrade ls --image quay.io/ceph/ceph --tags | grep v16.2.2 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_07:44:22 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T08:05:56.886 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa; will attempt to use it 2024-09-16T08:05:56.886 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa/tasks 2024-09-16T08:05:56.887 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T08:05:56.888 INFO:teuthology.task.internal:Checking packages... 2024-09-16T08:05:56.911 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8293d73f8690540e843a81caec373f9cc29cf705' 2024-09-16T08:05:56.911 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T08:05:56.911 INFO:teuthology.packaging:ref: None 2024-09-16T08:05:56.911 INFO:teuthology.packaging:tag: None 2024-09-16T08:05:56.911 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-13-1248 2024-09-16T08:05:56.911 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:05:56.912 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-13-1248 2024-09-16T08:05:57.058 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4917.g8293d73f 2024-09-16T08:05:57.060 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T08:05:57.071 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T08:05:57.072 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T08:05:57.105 INFO:teuthology.task.internal:Saving configuration 2024-09-16T08:05:57.117 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T08:05:57.164 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T08:05:57.190 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 08:00:10.515551', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKJBHN4PORr3u0gCJNBcZXYFQVc4sQCOyHsxGRxuz2oaRxWLuS1R6AUDoyCAPwOZ3L/8anKbT5v4e1pHMKBW3RQ='} 2024-09-16T08:05:57.212 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 08:00:10.517304', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFI3q8MO/o/QqKIi0uyIf+VkDGR6NWb3qnObNsRWyMCIMfoZyuXgySPpDqcLxoDSQ1VMVI2Yp5ovbSPSjoyC2oM='} 2024-09-16T08:05:57.212 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T08:05:57.224 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2024-09-16T08:05:57.224 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-09-16T08:05:57.224 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T08:05:57.308 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7efcb26ffbe0>, signals=[15]) 2024-09-16T08:05:57.308 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T08:05:57.319 INFO:teuthology.task.internal:Opening connections... 2024-09-16T08:05:57.319 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-09-16T08:05:57.321 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:05:57.398 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-09-16T08:05:57.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:05:57.476 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T08:05:57.488 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-09-16T08:05:57.506 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-09-16T08:05:57.506 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-09-16T08:05:57.563 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-09-16T08:05:57.563 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-09-16T08:05:57.563 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-09-16T08:05:57.563 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-09-16T08:05:57.563 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-09-16T08:05:57.563 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-09-16T08:05:57.564 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T08:05:57.564 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-09-16T08:05:57.564 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-09-16T08:05:57.564 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T08:05:57.564 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-09-16T08:05:57.564 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T08:05:57.564 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T08:05:57.564 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T08:05:57.565 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-09-16T08:05:57.588 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-09-16T08:05:57.607 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-09-16T08:05:57.607 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-09-16T08:05:57.665 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-09-16T08:05:57.665 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T08:05:57.666 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T08:05:57.667 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-09-16T08:05:57.687 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T08:05:57.699 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T08:05:57.760 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T08:05:57.761 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-09-16T08:05:57.763 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-09-16T08:05:57.780 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T08:05:57.810 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T08:05:57.810 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-09-16T08:05:57.821 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-09-16T08:05:57.844 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T08:05:57.906 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T08:05:57.924 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T08:05:57.924 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T08:05:57.925 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T08:05:57.925 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T08:05:57.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:05:57.942 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-09-16T08:05:57.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:05:57.943 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-09-16T08:05:57.999 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-503.el9.x86_64 2024-09-16T08:05:58.000 INFO:teuthology.task.kernel:Running kernel on smithi193: 5.14.0-503.el9.x86_64 2024-09-16T08:05:58.000 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2024-09-16T08:05:58.001 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-503.el9.x86_64 2024-09-16T08:05:58.001 INFO:teuthology.task.kernel:Running kernel on smithi090: 5.14.0-503.el9.x86_64 2024-09-16T08:05:58.001 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum install -y kernel 2024-09-16T08:06:00.508 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-16T08:06:01.953 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-16T08:06:04.759 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 20 MB 00:03 2024-09-16T08:06:04.796 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-16T08:06:09.051 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-16T08:06:09.306 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.5 MB 00:00 2024-09-16T08:06:10.504 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-16T08:06:10.741 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 92 kB/s | 19 kB 00:00 2024-09-16T08:06:11.606 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-16T08:06:11.738 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-16T08:06:16.995 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T08:06:17.178 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T08:06:19.010 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T08:06:19.010 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T08:06:19.108 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-09-16T08:06:19.118 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-09-16T08:06:19.119 INFO:teuthology.orchestra.run.smithi193.stdout:Install 4 Packages 2024-09-16T08:06:19.120 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:06:19.120 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 79 M 2024-09-16T08:06:19.120 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 126 M 2024-09-16T08:06:19.120 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-09-16T08:06:19.218 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T08:06:19.218 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T08:06:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-09-16T08:06:19.344 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-09-16T08:06:19.345 INFO:teuthology.orchestra.run.smithi090.stdout:Install 4 Packages 2024-09-16T08:06:19.346 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:06:19.346 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 79 M 2024-09-16T08:06:19.346 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 126 M 2024-09-16T08:06:19.346 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-09-16T08:06:19.463 INFO:teuthology.orchestra.run.smithi193.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 358 kB/s | 23 kB 00:00 2024-09-16T08:06:19.588 INFO:teuthology.orchestra.run.smithi090.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 219 kB/s | 23 kB 00:00 2024-09-16T08:06:20.005 INFO:teuthology.orchestra.run.smithi193.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-16T08:06:20.172 INFO:teuthology.orchestra.run.smithi090.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 49 MB/s | 29 MB 00:00 2024-09-16T08:06:20.472 INFO:teuthology.orchestra.run.smithi193.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 32 MB/s | 35 MB 00:01 2024-09-16T08:06:20.639 INFO:teuthology.orchestra.run.smithi090.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-09-16T08:06:20.963 INFO:teuthology.orchestra.run.smithi193.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 19 MB/s | 29 MB 00:01 2024-09-16T08:06:20.964 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:06:20.964 INFO:teuthology.orchestra.run.smithi193.stdout:Total 43 MB/s | 79 MB 00:01 2024-09-16T08:06:21.023 INFO:teuthology.orchestra.run.smithi090.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 22 MB/s | 35 MB 00:01 2024-09-16T08:06:21.023 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:06:21.023 INFO:teuthology.orchestra.run.smithi090.stdout:Total 47 MB/s | 79 MB 00:01 2024-09-16T08:06:21.652 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-09-16T08:06:21.706 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-09-16T08:06:21.993 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-09-16T08:06:21.993 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-09-16T08:06:22.065 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-09-16T08:06:22.065 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-09-16T08:06:22.957 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-09-16T08:06:22.958 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-09-16T08:06:22.966 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-09-16T08:06:22.966 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-09-16T08:06:25.407 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-09-16T08:06:25.481 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-09-16T08:06:26.098 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:06:26.172 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:06:26.180 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:06:26.255 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:06:28.963 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:06:29.025 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:06:29.092 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:06:29.173 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:06:33.925 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:06:33.959 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:06:34.018 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:06:34.059 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:06:38.830 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:06:38.957 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:07:16.951 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:07:16.990 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:07:17.349 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:07:17.349 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:07:17.349 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:07:17.349 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:07:18.210 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:07:18.211 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:07:18.211 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-09-16T08:07:18.211 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T08:07:18.211 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T08:07:18.211 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T08:07:18.211 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T08:07:18.211 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:07:18.211 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-09-16T08:07:18.576 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:07:18.662 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:07:18.662 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:07:18.662 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T08:07:18.662 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T08:07:18.663 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T08:07:18.663 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T08:07:18.663 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi193.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T08:07:18.663 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:07:18.721 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:07:18.722 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:07:18.722 DEBUG:teuthology.orchestra.run.smithi193:> sudo rpm -qi grub2-tools 2024-09-16T08:07:18.754 INFO:teuthology.orchestra.run.smithi193.stdout:Name : grub2-tools 2024-09-16T08:07:18.754 INFO:teuthology.orchestra.run.smithi193.stdout:Epoch : 1 2024-09-16T08:07:18.754 INFO:teuthology.orchestra.run.smithi193.stdout:Version : 2.06 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Release : 82.el9 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Architecture: x86_64 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Group : Unspecified 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Size : 8274670 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:License : GPLv3+ 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Packager : builder@centos.org 2024-09-16T08:07:18.755 INFO:teuthology.orchestra.run.smithi193.stdout:Vendor : CentOS 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout:Summary : Support tools for GRUB. 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout:Description : 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout:hardware devices. 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:07:18.756 INFO:teuthology.orchestra.run.smithi193.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T08:07:18.757 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T08:07:18.757 INFO:teuthology.task.kernel:Updating grub on smithi193 to boot 5.14.0-505.el9.x86_64 2024-09-16T08:07:18.758 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T08:07:19.022 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:07:19.062 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:07:19.409 INFO:teuthology.orchestra.run.smithi193.stderr:Generating grub configuration file ... 2024-09-16T08:07:19.528 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:07:19.529 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:07:19.529 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:07:19.529 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:07:20.468 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:07:20.468 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:07:20.468 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-09-16T08:07:20.468 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T08:07:20.468 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T08:07:20.468 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T08:07:20.468 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T08:07:20.468 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:07:20.468 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-09-16T08:07:20.716 INFO:teuthology.orchestra.run.smithi193.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T08:07:20.737 INFO:teuthology.orchestra.run.smithi193.stderr:done 2024-09-16T08:07:20.739 DEBUG:teuthology.orchestra.run.smithi193:> mktemp 2024-09-16T08:07:20.755 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.V3qYHx6Cjv 2024-09-16T08:07:20.755 DEBUG:teuthology.orchestra.run.smithi193:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.V3qYHx6Cjv 2024-09-16T08:07:20.823 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.V3qYHx6Cjv 2024-09-16T08:07:20.836 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:07:20.984 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.V3qYHx6Cjv is 6KB 2024-09-16T08:07:20.986 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:07:20.986 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:07:20.986 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T08:07:20.986 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T08:07:20.986 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T08:07:20.987 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T08:07:20.987 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T08:07:20.987 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:07:21.034 DEBUG:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.V3qYHx6Cjv 2024-09-16T08:07:21.046 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:07:21.047 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:07:21.047 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -qi grub2-tools 2024-09-16T08:07:21.049 DEBUG:teuthology.orchestra.run.smithi193:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T08:07:21.078 INFO:teuthology.orchestra.run.smithi090.stdout:Name : grub2-tools 2024-09-16T08:07:21.078 INFO:teuthology.orchestra.run.smithi090.stdout:Epoch : 1 2024-09-16T08:07:21.078 INFO:teuthology.orchestra.run.smithi090.stdout:Version : 2.06 2024-09-16T08:07:21.078 INFO:teuthology.orchestra.run.smithi090.stdout:Release : 82.el9 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: x86_64 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Group : Unspecified 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Size : 8274670 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:License : GPLv3+ 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Packager : builder@centos.org 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Vendor : CentOS 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Summary : Support tools for GRUB. 2024-09-16T08:07:21.079 INFO:teuthology.orchestra.run.smithi090.stdout:Description : 2024-09-16T08:07:21.080 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:07:21.080 INFO:teuthology.orchestra.run.smithi090.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T08:07:21.080 INFO:teuthology.orchestra.run.smithi090.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T08:07:21.080 INFO:teuthology.orchestra.run.smithi090.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T08:07:21.080 INFO:teuthology.orchestra.run.smithi090.stdout:hardware devices. 2024-09-16T08:07:21.080 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:07:21.080 INFO:teuthology.orchestra.run.smithi090.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T08:07:21.081 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T08:07:21.081 INFO:teuthology.task.kernel:Updating grub on smithi090 to boot 5.14.0-505.el9.x86_64 2024-09-16T08:07:21.081 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T08:07:21.112 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T08:07:21.112 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T08:07:21.112 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T08:07:21.112 INFO:teuthology.orchestra.run.smithi193.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T08:07:21.113 DEBUG:teuthology.orchestra.run.smithi193:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T08:07:21.267 DEBUG:teuthology.orchestra.run.smithi193:> sudo shutdown -r now 2024-09-16T08:07:21.743 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2024-09-16T08:07:23.450 INFO:teuthology.orchestra.run.smithi090.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T08:07:23.472 INFO:teuthology.orchestra.run.smithi090.stderr:done 2024-09-16T08:07:23.474 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-09-16T08:07:23.494 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.FKx6yLpPFR 2024-09-16T08:07:23.495 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FKx6yLpPFR 2024-09-16T08:07:23.561 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.FKx6yLpPFR 2024-09-16T08:07:23.707 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.FKx6yLpPFR is 6KB 2024-09-16T08:07:23.755 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.FKx6yLpPFR 2024-09-16T08:07:23.781 DEBUG:teuthology.orchestra.run.smithi090:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T08:07:23.840 INFO:teuthology.orchestra.run.smithi090.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T08:07:23.840 INFO:teuthology.orchestra.run.smithi090.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T08:07:23.840 INFO:teuthology.orchestra.run.smithi090.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T08:07:23.840 INFO:teuthology.orchestra.run.smithi090.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T08:07:23.842 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T08:07:24.003 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2024-09-16T08:07:51.297 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T08:07:51.298 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-16T08:07:51.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:07:54.007 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T08:07:54.007 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-16T08:07:54.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:08:09.789 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-09-16T08:08:12.349 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-09-16T08:08:18.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-16T08:08:18.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:08:21.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-16T08:08:21.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:08:21.857 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.193 2024-09-16T08:08:24.417 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-09-16T08:08:33.867 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-16T08:08:33.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:08:36.421 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-16T08:08:36.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:08:39.485 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-09-16T08:08:54.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-16T08:08:54.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:09:05.697 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-16T08:09:06.329 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-16T08:09:06.330 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T08:09:06.330 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-09-16T08:09:06.347 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-505.el9.x86_64 2024-09-16T08:09:06.347 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T08:09:06.347 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T08:09:06.347 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T08:09:07.348 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T08:09:07.348 DEBUG:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T08:09:07.409 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2024-09-16T08:09:07.436 DEBUG:teuthology.parallel:result is None 2024-09-16T08:09:13.025 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-09-16T08:09:31.026 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-16T08:09:31.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:09:31.521 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-09-16T08:09:31.918 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-16T08:09:31.918 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T08:09:31.919 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-09-16T08:09:31.933 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-505.el9.x86_64 2024-09-16T08:09:31.934 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T08:09:31.934 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T08:09:31.934 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T08:09:32.934 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T08:09:32.935 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T08:09:33.096 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2024-09-16T08:09:33.125 DEBUG:teuthology.parallel:result is None 2024-09-16T08:09:33.125 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T08:09:33.132 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T08:09:33.132 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T08:09:33.134 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T08:09:33.169 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T08:09:33.175 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T08:09:33.194 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T08:09:33.194 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T08:09:33.208 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T08:09:33.272 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T08:09:33.278 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T08:09:33.279 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T08:09:33.285 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T08:09:33.319 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:09:33.331 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:09:33.361 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:09:33.384 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:09:33.386 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T08:09:33.396 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T08:09:33.396 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T08:09:33.399 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T08:09:33.454 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T08:09:33.463 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T08:09:33.463 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T08:09:33.466 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T08:09:33.502 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:09:33.602 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:09:33.649 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:09:33.730 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:09:33.767 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:09:33.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T08:09:33.830 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:09:33.872 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:09:33.931 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:09:34.017 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:09:34.062 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T08:09:34.062 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T08:09:34.131 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-09-16T08:09:34.134 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-09-16T08:09:34.226 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T08:09:34.305 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T08:09:34.541 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T08:09:34.551 INFO:teuthology.task.internal:Starting timer... 2024-09-16T08:09:34.551 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T08:09:34.691 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T08:09:34.757 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-09-16T08:09:34.877 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-09-16T08:09:34.879 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-09-16T08:09:34.934 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-09-16T08:09:34.937 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T08:09:34.937 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-09-16T08:09:34.957 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-09-16T08:09:34.957 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-09-16T08:09:34.978 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-09-16T08:09:34.978 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-09-16T08:09:34.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T08:09:35.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:09:35.022 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T08:09:35.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:09:35.051 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T08:09:35.051 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-09-16T08:09:35.096 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-09-16T08:09:35.129 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T08:09:35.139 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T08:09:35.142 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T08:09:35.261 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T08:09:35.272 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T08:09:35.273 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T08:13:23.480 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-09-16T08:13:23.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-16T08:13:23.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:13:23.561 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-09-16T08:13:23.644 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-16T08:13:23.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-16T08:13:23.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:13:23.719 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-09-16T08:13:23.796 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-09-16T08:13:23.797 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T08:13:23.809 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T08:13:23.810 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T08:13:23.811 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T08:13:23.814 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T08:13:23.814 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T08:13:23.841 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T08:13:23.856 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T08:13:23.866 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T08:13:23.880 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-09-16T08:13:23.881 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T08:13:23.892 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-09-16T08:13:23.905 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-09-16T08:13:23.905 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T08:13:23.916 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-09-16T08:13:23.920 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T08:13:23.931 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T08:13:23.944 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T08:13:23.977 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-09-16T08:13:23.981 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T08:13:23.981 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-09-16T08:13:23.981 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:13:23.981 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:13:23.981 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:13:23.981 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:13:24.004 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-16T08:13:24.008 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T08:13:24.008 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-16T08:13:24.008 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:13:24.008 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:13:24.008 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:13:24.008 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:13:24.009 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T08:13:24.116 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'conf': {'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'CEPHADM_AGENT_DOWN'], 'log-only-match': ['CEPHADM_'], 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'} 2024-09-16T08:13:24.116 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-09-16T08:13:24.117 INFO:tasks.cephadm:Cluster fsid is 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:13:24.117 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T08:13:24.118 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.90', 'mon.c': '[v2:172.21.15.90:3301,v1:172.21.15.90:6790]', 'mon.b': '172.21.15.193'} 2024-09-16T08:13:24.118 INFO:tasks.cephadm:First mon is mon.a on smithi090 2024-09-16T08:13:24.118 INFO:tasks.cephadm:First mgr is y 2024-09-16T08:13:24.118 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T08:13:24.119 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-09-16T08:13:24.145 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-09-16T08:13:24.171 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-09-16T08:13:24.172 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:13:24.317 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83046/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4917-g8293d73f', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4917.g8293d73f'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-13 15:55:14.480698', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-main-2024-09-13-1248', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-16T08:13:24.581 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 ae7f37330c1a72493684eb1fcaed6f2a01195bb6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-09-16T08:13:24.583 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/x86_64/flavors/default/cephadm 2024-09-16T08:13:24.583 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/x86_64/flavors/default/cephadm 2024-09-16T08:13:24.583 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T08:13:24.918 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 217225 Sep 16 08:13 /home/ubuntu/cephtest/cephadm 2024-09-16T08:13:24.919 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T08:13:25.263 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 217225 Sep 16 08:13 /home/ubuntu/cephtest/cephadm 2024-09-16T08:13:25.263 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T08:13:25.283 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T08:13:25.317 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-09-16T08:13:25.317 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-09-16T08:13:25.325 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-09-16T08:13:26.957 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-09-16T08:13:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-09-16T08:14:04.198 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-09-16T08:14:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)", 2024-09-16T08:14:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95", 2024-09-16T08:14:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-09-16T08:14:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87" 2024-09-16T08:14:04.199 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-09-16T08:14:04.199 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-09-16T08:14:08.596 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:14:08.596 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)", 2024-09-16T08:14:08.596 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95", 2024-09-16T08:14:08.596 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-09-16T08:14:08.597 INFO:teuthology.orchestra.run.smithi090.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87" 2024-09-16T08:14:08.597 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-16T08:14:08.597 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:14:08.619 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-09-16T08:14:08.654 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-09-16T08:14:08.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-09-16T08:14:08.720 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-09-16T08:14:08.756 INFO:tasks.cephadm:Writing seed config... 2024-09-16T08:14:08.757 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-16T08:14:08.757 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T08:14:08.757 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T08:14:08.757 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2024-09-16T08:14:08.757 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T08:14:08.757 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T08:14:08.758 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T08:14:08.758 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T08:14:08.758 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T08:14:08.759 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:14:08.759 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T08:14:08.777 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8b378aa4-7403-11ef-bceb-c7b262605968 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-16T08:14:08.777 DEBUG:teuthology.orchestra.run.smithi090:mon.a> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T08:14:08.819 DEBUG:teuthology.orchestra.run.smithi090:mgr.y> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T08:14:08.863 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T08:14:08.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.90 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:14:09.015 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:14:09.015 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', '8b378aa4-7403-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.90', '--skip-admin-label'] 2024-09-16T08:14:09.038 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 5.2.2 2024-09-16T08:14:09.039 INFO:teuthology.orchestra.run.smithi090.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T08:14:09.039 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying podman|docker is present... 2024-09-16T08:14:09.066 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 5.2.2 2024-09-16T08:14:09.067 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying lvm2 is present... 2024-09-16T08:14:09.067 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying time synchronization is in place... 2024-09-16T08:14:09.074 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T08:14:09.074 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T08:14:09.080 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T08:14:09.081 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-16T08:14:09.087 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-09-16T08:14:09.093 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-09-16T08:14:09.093 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-09-16T08:14:09.093 INFO:teuthology.orchestra.run.smithi090.stdout:Repeating the final host check... 2024-09-16T08:14:09.115 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 5.2.2 2024-09-16T08:14:09.115 INFO:teuthology.orchestra.run.smithi090.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T08:14:09.115 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl is present 2024-09-16T08:14:09.115 INFO:teuthology.orchestra.run.smithi090.stdout:lvcreate is present 2024-09-16T08:14:09.121 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T08:14:09.122 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T08:14:09.128 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T08:14:09.128 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-16T08:14:09.134 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-09-16T08:14:09.141 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-09-16T08:14:09.141 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-09-16T08:14:09.141 INFO:teuthology.orchestra.run.smithi090.stdout:Host looks OK 2024-09-16T08:14:09.141 INFO:teuthology.orchestra.run.smithi090.stdout:Cluster fsid: 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:09.141 INFO:teuthology.orchestra.run.smithi090.stdout:Acquiring lock 140099949527200 on /run/cephadm/8b378aa4-7403-11ef-bceb-c7b262605968.lock 2024-09-16T08:14:09.141 INFO:teuthology.orchestra.run.smithi090.stdout:Lock 140099949527200 acquired on /run/cephadm/8b378aa4-7403-11ef-bceb-c7b262605968.lock 2024-09-16T08:14:09.141 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 3300 ... 2024-09-16T08:14:09.142 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 6789 ... 2024-09-16T08:14:09.142 INFO:teuthology.orchestra.run.smithi090.stdout:Base mon IP(s) is [172.21.15.90:3300, 172.21.15.90:6789], mon addrv is [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-09-16T08:14:09.145 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.90 metric 100 2024-09-16T08:14:09.145 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.90 metric 100 2024-09-16T08:14:09.147 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T08:14:09.147 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T08:14:09.147 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T08:14:09.149 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T08:14:09.150 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T08:14:09.150 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T08:14:09.150 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T08:14:09.150 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d767/64 scope link noprefixroute 2024-09-16T08:14:09.150 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T08:14:09.151 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-09-16T08:14:09.151 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-09-16T08:14:09.151 INFO:teuthology.orchestra.run.smithi090.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T08:14:09.151 INFO:teuthology.orchestra.run.smithi090.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T08:14:09.151 INFO:teuthology.orchestra.run.smithi090.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-09-16T08:14:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 2024-09-16T08:14:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-09-16T08:14:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T08:14:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:7554792ee943fcbd9ac63c8a3057c4d86592e594de711470628678498ff1ba71 2024-09-16T08:14:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T08:14:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying config sha256:fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 2024-09-16T08:14:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T08:14:10.482 INFO:teuthology.orchestra.run.smithi090.stdout:ceph: stdout ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable) 2024-09-16T08:14:10.482 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph version: ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable) 2024-09-16T08:14:10.482 INFO:teuthology.orchestra.run.smithi090.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T08:14:11.339 INFO:teuthology.orchestra.run.smithi090.stdout:stat: stdout 167 167 2024-09-16T08:14:11.339 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial keys... 2024-09-16T08:14:12.180 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQDT6Odm0Jw1JhAAcAMxbGMilqSl0NVTids/qQ== 2024-09-16T08:14:13.046 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQDU6OdmQrErHRAA7BMTjsCpGTXQ1qydt6rWaA== 2024-09-16T08:14:14.054 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQDV6OdmA/I/FRAAUPuJEcg5Rv0R0bS7nelctg== 2024-09-16T08:14:14.055 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial monmap... 2024-09-16T08:14:14.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T08:14:14.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-16T08:14:14.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:14.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T08:14:14.905 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool for a [v2:172.21.15.90:3300,v1:172.21.15.90:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T08:14:14.905 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = pacific 2024-09-16T08:14:14.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: set fsid to 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:14.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T08:14:14.906 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:14:14.906 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mon... 2024-09-16T08:14:15.821 INFO:teuthology.orchestra.run.smithi090.stdout:create mon.a on 2024-09-16T08:14:16.184 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T08:14:16.380 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8b378aa4-7403-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-8b378aa4-7403-11ef-bceb-c7b262605968.target. 2024-09-16T08:14:16.380 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8b378aa4-7403-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-8b378aa4-7403-11ef-bceb-c7b262605968.target. 2024-09-16T08:14:16.609 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.a 2024-09-16T08:14:16.609 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T08:14:16.777 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8b378aa4-7403-11ef-bceb-c7b262605968.target.wants/ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-8b378aa4-7403-11ef-bceb-c7b262605968@.service. 2024-09-16T08:14:17.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:17 smithi090 podman[23320]: 2024-09-16 08:14:16.917428416 +0000 UTC m=+0.015558033 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph:reef 2024-09-16T08:14:17.300 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:14:17.300 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-09-16T08:14:17.308 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:14:17.308 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-16T08:14:17.308 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-09-16T08:14:17.308 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T08:14:17.308 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon to start... 2024-09-16T08:14:17.309 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon... 2024-09-16T08:14:17.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:17 smithi090 podman[23320]: 2024-09-16 08:14:17.209628516 +0000 UTC m=+0.307758119 container init 37d94f7cb1f1507ea1bdd34df2dfdfa7117c942d8f0bab370aec56cd87c8894f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T08:14:17.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:17 smithi090 podman[23320]: 2024-09-16 08:14:17.21293579 +0000 UTC m=+0.311065395 container start 37d94f7cb1f1507ea1bdd34df2dfdfa7117c942d8f0bab370aec56cd87c8894f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, RELEASE=reef-ae7f373, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T08:14:17.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:17 smithi090 ceph-mon[23338]: mkfs 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:17.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:17 smithi090 ceph-mon[23338]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T08:14:17.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:17 smithi090 bash[23320]: 37d94f7cb1f1507ea1bdd34df2dfdfa7117c942d8f0bab370aec56cd87c8894f 2024-09-16T08:14:17.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:17 smithi090 systemd[1]: Started Ceph mon.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:14:18.298 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout id: 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout services: 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.530754s) 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout data: 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T08:14:18.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T08:14:18.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T08:14:18.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T08:14:18.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:18.300 INFO:teuthology.orchestra.run.smithi090.stdout:mon is available 2024-09-16T08:14:18.300 INFO:teuthology.orchestra.run.smithi090.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T08:14:18.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:18 smithi090 ceph-mon[23338]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T08:14:18.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:18 smithi090 ceph-mon[23338]: monmap e1: 1 mons at {a=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:14:18.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:18 smithi090 ceph-mon[23338]: fsmap 2024-09-16T08:14:18.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:18 smithi090 ceph-mon[23338]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T08:14:18.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:18 smithi090 ceph-mon[23338]: mgrmap e1: no daemons active 2024-09-16T08:14:18.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:18 smithi090 ceph-mon[23338]: from='client.? 172.21.15.90:0/1571843134' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T08:14:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T08:14:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-09-16T08:14:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T08:14:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T08:14:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T08:14:19.331 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T08:14:19.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:19.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T08:14:19.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = True 2024-09-16T08:14:19.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T08:14:19.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:19.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T08:14:19.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T08:14:19.332 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T08:14:19.332 INFO:teuthology.orchestra.run.smithi090.stdout:Generating new minimal ceph.conf... 2024-09-16T08:14:19.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:19 smithi090 ceph-mon[23338]: from='client.? 172.21.15.90:0/34552492' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T08:14:19.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:19 smithi090 ceph-mon[23338]: from='client.? 172.21.15.90:0/34552492' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T08:14:20.297 INFO:teuthology.orchestra.run.smithi090.stdout:Restarting the monitor... 2024-09-16T08:14:20.557 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:20 smithi090 ceph-mon[23338]: from='client.? 172.21.15.90:0/2394338208' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:14:20.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:20 smithi090 systemd[1]: Stopping Ceph mon.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:14:20.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:20 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a[23334]: 2024-09-16T08:14:20.556+0000 7f1e46bbd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:14:20.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:20 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a[23334]: 2024-09-16T08:14:20.556+0000 7f1e46bbd640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T08:14:20.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:20 smithi090 podman[23579]: 2024-09-16 08:14:20.77086124 +0000 UTC m=+0.316189685 container died 37d94f7cb1f1507ea1bdd34df2dfdfa7117c942d8f0bab370aec56cd87c8894f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T08:14:21.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 podman[23579]: 2024-09-16 08:14:21.099221531 +0000 UTC m=+0.644549931 container cleanup 37d94f7cb1f1507ea1bdd34df2dfdfa7117c942d8f0bab370aec56cd87c8894f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T08:14:21.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 bash[23579]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a 2024-09-16T08:14:21.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 podman[23591]: 2024-09-16 08:14:21.283603314 +0000 UTC m=+0.505459736 container remove 37d94f7cb1f1507ea1bdd34df2dfdfa7117c942d8f0bab370aec56cd87c8894f (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-09-16T08:14:21.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T08:14:21.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 systemd[1]: Stopped Ceph mon.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:14:21.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 systemd[1]: Starting Ceph mon.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:14:22.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 podman[23682]: 2024-09-16 08:14:21.72642385 +0000 UTC m=+0.103570586 container create 87c2b7a329d6c2aae532fd9ba310cfa1379a04fe15e15c99df3f3e188e387f6d (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:14:22.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 podman[23682]: 2024-09-16 08:14:21.638696211 +0000 UTC m=+0.015842907 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph:reef 2024-09-16T08:14:22.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 podman[23682]: 2024-09-16 08:14:21.937335116 +0000 UTC m=+0.314481819 container init 87c2b7a329d6c2aae532fd9ba310cfa1379a04fe15e15c99df3f3e188e387f6d (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T08:14:22.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 podman[23682]: 2024-09-16 08:14:21.940430604 +0000 UTC m=+0.317577301 container start 87c2b7a329d6c2aae532fd9ba310cfa1379a04fe15e15c99df3f3e188e387f6d (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T08:14:22.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable), process ceph-mon, pid 2 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: pidfile_write: ignore empty --pid-file 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: load: jerasure load: lrc 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: RocksDB version: 7.9.2 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Git sha 0 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Compile date 2024-09-15 20:36:04 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: DB SUMMARY 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: DB Session ID: 75Z986TTA8BKO9OJ3WER 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: CURRENT file: CURRENT 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 83630 ; 2024-09-16T08:14:22.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.error_if_exists: 0 2024-09-16T08:14:22.014 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.create_if_missing: 0 2024-09-16T08:14:22.014 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.paranoid_checks: 1 2024-09-16T08:14:22.014 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.env: 0x565374bdebc0 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.info_log: 0x565377276be0 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.statistics: (nil) 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.use_fsync: 0 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_log_file_size: 0 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T08:14:22.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.allow_fallocate: 1 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.use_direct_reads: 0 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.db_log_dir: 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.wal_dir: 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T08:14:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.write_buffer_manager: 0x565377284320 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T08:14:22.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.unordered_write: 0 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.row_cache: None 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.wal_filter: None 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.two_write_queues: 0 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T08:14:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.wal_compression: 0 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.atomic_flush: 0 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.log_readahead_size: 0 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_background_jobs: 2 2024-09-16T08:14:22.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_background_compactions: -1 2024-09-16T08:14:22.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_subcompactions: 1 2024-09-16T08:14:22.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T08:14:22.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T08:14:22.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T08:14:22.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T08:14:22.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T08:14:22.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T08:14:22.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T08:14:22.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T08:14:22.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_open_files: -1 2024-09-16T08:14:22.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T08:14:22.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T08:14:22.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T08:14:22.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T08:14:22.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_background_flushes: -1 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Compression algorithms supported: 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: kZSTD supported: 0 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: kXpressCompression supported: 0 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: kBZip2Compression supported: 0 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: kLZ4Compression supported: 1 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: kZlibCompression supported: 1 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: kSnappyCompression supported: 1 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T08:14:22.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.merge_operator: 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_filter: None 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5653772767e0) 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-09-16T08:14:22.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_type: 0 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_index_type: 0 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_shortening: 1 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: checksum: 4 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: no_block_cache: 0 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache: 0x56537726b350 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_options: 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: capacity : 536870912 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_shard_bits : 4 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: strict_capacity_limit : 0 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-09-16T08:14:22.024 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_compressed: (nil) 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: persistent_cache: (nil) 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size: 4096 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size_deviation: 10 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_restart_interval: 16 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_block_restart_interval: 1 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: metadata_block_size: 4096 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: partition_filters: 0 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: use_delta_encoding: 1 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: filter_policy: bloomfilter 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: whole_key_filtering: 1 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: verify_compression: 0 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-09-16T08:14:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout: format_version: 5 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_index_compression: 1 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_align: 0 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout: max_auto_readahead_size: 262144 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout: prepopulate_block_cache: 0 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout: initial_auto_readahead_size: 8192 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression: NoCompression 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.num_levels: 7 2024-09-16T08:14:22.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:14:22.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T08:14:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T08:14:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T08:14:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T08:14:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T08:14:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T08:14:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T08:14:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T08:14:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T08:14:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T08:14:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T08:14:22.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.inplace_update_support: 0 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T08:14:22.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T08:14:22.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T08:14:22.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.bloom_locality: 0 2024-09-16T08:14:22.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.max_successive_merges: 0 2024-09-16T08:14:22.032 INFO:teuthology.orchestra.run.smithi090.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.ttl: 2592000 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.enable_blob_files: false 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.min_blob_size: 0 2024-09-16T08:14:22.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e1fd6e17-a1a0-4c0f-a584-03c702246fc7 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726474461966573, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T08:14:22.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726474461967409, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 80279, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 252, "table_properties": {"data_size": 78440, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 72751, "raw_average_value_size": 344, "num_data_blocks": 10, "num_entries": 211, "num_filter_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726474461, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e1fd6e17-a1a0-4c0f-a584-03c702246fc7", "db_session_id": "75Z986TTA8BKO9OJ3WER", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726474461967503, "job": 1, "event": "recovery_finished"} 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x565377296e00 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: DB pointer 0x56537738a000 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** DB Stats ** 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:14:22.035 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: L0 2/0 80.26 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: Sum 2/0 80.26 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-09-16T08:14:22.036 INFO:journalctl@ceph.mon.a.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T08:14:22.037 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative compaction: 0.00 GB write, 13.75 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval compaction: 0.00 GB write, 13.75 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache BinnedLRUCache@0x56537726b350#2 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: starting mon.a rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: mon.a@-1(???) e1 preinit fsid 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: mon.a@-1(???).mds e1 new map 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: mon.a@-1(???).mds e1 print_map 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout: e1 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:14:22.038 INFO:journalctl@ceph.mon.a.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout: legacy client fscid: -1 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout: No filesystems configured 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:21 smithi090 ceph-mon[23701]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:22 smithi090 bash[23682]: 87c2b7a329d6c2aae532fd9ba310cfa1379a04fe15e15c99df3f3e188e387f6d 2024-09-16T08:14:22.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:22 smithi090 systemd[1]: Started Ceph mon.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:14:22.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:22 smithi090 ceph-mon[23701]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T08:14:22.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:22 smithi090 ceph-mon[23701]: monmap e1: 1 mons at {a=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:14:22.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:22 smithi090 ceph-mon[23701]: fsmap 2024-09-16T08:14:22.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:22 smithi090 ceph-mon[23701]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T08:14:22.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:22 smithi090 ceph-mon[23701]: mgrmap e1: no daemons active 2024-09-16T08:14:22.940 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T08:14:22.961 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:14:22.962 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mgr... 2024-09-16T08:14:22.962 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T08:14:22.962 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T08:14:23.206 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y 2024-09-16T08:14:23.206 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-16T08:14:23.356 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8b378aa4-7403-11ef-bceb-c7b262605968.target.wants/ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-8b378aa4-7403-11ef-bceb-c7b262605968@.service. 2024-09-16T08:14:23.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:23 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2467924859' entity='client.admin' 2024-09-16T08:14:23.885 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:23 smithi090 podman[23913]: 2024-09-16 08:14:23.796839983 +0000 UTC m=+0.326129217 container start 92b236ae51c08f45e454c3c46957c820b9993f2ff2d91b197b032c09d0795668 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=reef-ae7f373, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:14:23.885 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:23 smithi090 bash[23913]: 92b236ae51c08f45e454c3c46957c820b9993f2ff2d91b197b032c09d0795668 2024-09-16T08:14:23.899 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:14:23.899 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-09-16T08:14:23.906 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:14:23.906 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-16T08:14:23.906 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-09-16T08:14:23.906 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T08:14:23.915 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:14:23.915 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-09-16T08:14:23.922 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:14:23.922 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-16T08:14:23.923 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-09-16T08:14:23.923 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T08:14:23.923 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr to start... 2024-09-16T08:14:23.923 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr... 2024-09-16T08:14:24.178 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:23 smithi090 systemd[1]: Started Ceph mgr.y for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:14:24.178 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:23 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:23.915+0000 7f4f94c4d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T08:14:24.179 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:23 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:23.976+0000 7f4f94c4d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T08:14:24.179 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:24 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:24.126+0000 7f4f94c4d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T08:14:24.434 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:24 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:24.195+0000 7f4f94c4d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T08:14:24.434 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:24 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:24.344+0000 7f4f94c4d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:14:24.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:24 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2686494293' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T08:14:24.804 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:24 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:24.491+0000 7f4f94c4d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T08:14:24.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:24.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:14:24.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "8b378aa4-7403-11ef-bceb-c7b262605968", 2024-09-16T08:14:24.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T08:14:24.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T08:14:24.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:24.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T08:14:24.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T08:14:24.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:24.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T08:14:24.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:24.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T08:14:24.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T08:14:24.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T08:14:24.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T08:14:24.979 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:24.980 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:14:24.981 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:14:24.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:24.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T08:14:24.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:24.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T08:14:17.251322+0000", 2024-09-16T08:14:24.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:14:24.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:24.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T08:14:24.982 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:14:24.982 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (1/15)... 2024-09-16T08:14:25.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:24 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:24.803+0000 7f4f94c4d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T08:14:25.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:24 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:24.864+0000 7f4f94c4d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T08:14:25.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:25 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:25.047+0000 7f4f94c4d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T08:14:26.311 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:25 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:25.902+0000 7f4f94c4d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T08:14:26.311 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:25 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:25.966+0000 7f4f94c4d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T08:14:26.923 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:26 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:26.668+0000 7f4f94c4d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T08:14:26.924 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:26 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:26.780+0000 7f4f94c4d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T08:14:26.924 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:26 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:26.923+0000 7f4f94c4d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:14:27.208 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:26 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:26.983+0000 7f4f94c4d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T08:14:27.209 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:27.139+0000 7f4f94c4d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T08:14:27.209 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:27.206+0000 7f4f94c4d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T08:14:27.460 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:27.269+0000 7f4f94c4d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T08:14:27.460 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:27.435+0000 7f4f94c4d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T08:14:27.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/287803699' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T08:14:27.720 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:27.510+0000 7f4f94c4d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T08:14:27.721 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:27.577+0000 7f4f94c4d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T08:14:27.721 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:27.658+0000 7f4f94c4d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T08:14:28.029 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:27.720+0000 7f4f94c4d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T08:14:28.029 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:27 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:27.780+0000 7f4f94c4d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "8b378aa4-7403-11ef-bceb-c7b262605968", 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T08:14:28.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T08:14:28.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T08:14:28.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:28.040 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T08:14:17.251322+0000", 2024-09-16T08:14:28.041 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:14:28.042 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:28.042 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T08:14:28.042 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:14:28.042 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (2/15)... 2024-09-16T08:14:28.811 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:28.556+0000 7f4f94c4d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: Activating manager daemon y 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: mgrmap e2: y(active, starting, since 0.00292444s) 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: Manager daemon y is now available 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:14:28.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' 2024-09-16T08:14:28.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' 2024-09-16T08:14:28.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:28 smithi090 ceph-mon[23701]: from='mgr.14100 172.21.15.90:0/2819610173' entity='mgr.y' 2024-09-16T08:14:30.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:30 smithi090 ceph-mon[23701]: mgrmap e3: y(active, since 1.00731s) 2024-09-16T08:14:31.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:31.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "8b378aa4-7403-11ef-bceb-c7b262605968", 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T08:14:31.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T08:14:31.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T08:14:31.157 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T08:14:31.157 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T08:14:31.157 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:31.157 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T08:14:31.157 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T08:14:31.157 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T08:14:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T08:14:17.251322+0000", 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:14:31.159 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T08:14:31.160 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:14:31.160 INFO:teuthology.orchestra.run.smithi090.stdout:mgr is available 2024-09-16T08:14:31.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:31 smithi090 ceph-mon[23701]: mgrmap e4: y(active, since 2s) 2024-09-16T08:14:31.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:31 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3753160377' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T08:14:32.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:32.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T08:14:32.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:32.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T08:14:32.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T08:14:32.168 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T08:14:32.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T08:14:32.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:32.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T08:14:32.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T08:14:32.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-16T08:14:32.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T08:14:32.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T08:14:32.169 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T08:14:32.169 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling cephadm module... 2024-09-16T08:14:32.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:32 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/831112533' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T08:14:32.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:32 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/831112533' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T08:14:33.916 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:33 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3341305891' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T08:14:33.917 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:33 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ignoring --setuser ceph since I am not root 2024-09-16T08:14:33.917 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:33 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ignoring --setgroup ceph since I am not root 2024-09-16T08:14:33.917 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:33 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:33.713+0000 7f4193276040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T08:14:33.917 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:33 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:33.770+0000 7f4193276040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T08:14:33.917 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:33 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:33.916+0000 7f4193276040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T08:14:34.268 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:33 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:33.981+0000 7f4193276040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T08:14:34.268 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:34 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:34.125+0000 7f4193276040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:14:34.268 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:34 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:34.262+0000 7f4193276040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T08:14:35.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:34 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3341305891' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T08:14:35.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:34 smithi090 ceph-mon[23701]: mgrmap e5: y(active, since 5s) 2024-09-16T08:14:35.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:34 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:34.566+0000 7f4193276040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T08:14:35.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:34 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:34.625+0000 7f4193276040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T08:14:35.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:34 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:34.807+0000 7f4193276040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T08:14:35.812 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:35 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:35.709+0000 7f4193276040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T08:14:36.258 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:35 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:35.778+0000 7f4193276040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T08:14:36.561 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:36.504+0000 7f4193276040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T08:14:36.966 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:36.620+0000 7f4193276040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T08:14:36.966 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:36.760+0000 7f4193276040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:14:36.967 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:36.820+0000 7f4193276040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T08:14:36.967 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:36.966+0000 7f4193276040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T08:14:37.249 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:37 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:37.035+0000 7f4193276040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T08:14:37.250 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:37 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:37.093+0000 7f4193276040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T08:14:37.250 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:37 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:37.249+0000 7f4193276040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T08:14:37.520 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:37 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:37.313+0000 7f4193276040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T08:14:37.520 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:37 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:37.379+0000 7f4193276040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T08:14:37.520 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:37 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:37.461+0000 7f4193276040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T08:14:37.520 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:37 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:37.520+0000 7f4193276040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T08:14:37.811 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:37 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:37.579+0000 7f4193276040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T08:14:38.811 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:38.351+0000 7f4193276040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: Active manager daemon y restarted 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: Activating manager daemon y 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: mgrmap e6: y(active, starting, since 0.00351359s) 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:14:38.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: Manager daemon y is now available 2024-09-16T08:14:38.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:14:38.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:14:38.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:14:38.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:14:38.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:14:38.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:14:38.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:38 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:14:39.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:39 smithi090 ceph-mon[23701]: Found migration_current of "None". Setting to last migration. 2024-09-16T08:14:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:39 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:14:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:39 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:14:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:39 smithi090 ceph-mon[23701]: mgrmap e7: y(active, since 1.00551s) 2024-09-16T08:14:40.312 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:40.057+0000 7f41007a1640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:08:14:40] ENGINE Error in HTTPServer.serve 2024-09-16T08:14:40.312 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: Traceback (most recent call last): 2024-09-16T08:14:40.312 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:14:40.312 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self._connections.run(self.expiration_interval) 2024-09-16T08:14:40.312 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self._run(expiration_interval) 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: s = self.context.wrap_socket( 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: return self.sslsocket_class._create( 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self.do_handshake() 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:14:40.313 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self._sslobj.do_handshake() 2024-09-16T08:14:40.314 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:14:40.314 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:14:40.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:40 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:14:41.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:41 smithi090 ceph-mon[23701]: [16/Sep/2024:08:14:39] ENGINE Bus STARTING 2024-09-16T08:14:41.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:41 smithi090 ceph-mon[23701]: [16/Sep/2024:08:14:40] ENGINE Serving on https://172.21.15.90:7150 2024-09-16T08:14:41.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:41 smithi090 ceph-mon[23701]: [16/Sep/2024:08:14:40] ENGINE Error in HTTPServer.serve 2024-09-16T08:14:41.812 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:14:41.812 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:14:41.812 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:14:41.812 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._run(expiration_interval) 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: s = self.context.wrap_socket( 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.sslsocket_class._create( 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:14:41.813 INFO:journalctl@ceph.mon.a.smithi090.stdout: self.do_handshake() 2024-09-16T08:14:41.814 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:14:41.814 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._sslobj.do_handshake() 2024-09-16T08:14:41.814 INFO:journalctl@ceph.mon.a.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:14:41.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:41 smithi090 ceph-mon[23701]: [16/Sep/2024:08:14:40] ENGINE Serving on http://172.21.15.90:8765 2024-09-16T08:14:41.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:41 smithi090 ceph-mon[23701]: [16/Sep/2024:08:14:40] ENGINE Bus STARTED 2024-09-16T08:14:41.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:41 smithi090 ceph-mon[23701]: mgrmap e8: y(active, since 2s) 2024-09-16T08:14:44.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:43 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2554669214' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T08:14:48.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:14:48.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-09-16T08:14:48.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T08:14:48.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T08:14:48.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T08:14:48.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:14:48.566 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-09-16T08:14:48.566 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 8... 2024-09-16T08:14:49.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:14:49.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-16T08:14:49.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T08:14:49.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:14:49.482 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 8 is available 2024-09-16T08:14:49.482 INFO:teuthology.orchestra.run.smithi090.stdout:Setting orchestrator backend to cephadm... 2024-09-16T08:14:49.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:49 smithi090 ceph-mon[23701]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T08:14:49.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:49 smithi090 ceph-mon[23701]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T08:14:51.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:51 smithi090 ceph-mon[23701]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:14:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:51 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:14:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:51 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:14:51.707 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T08:14:51.707 INFO:teuthology.orchestra.run.smithi090.stdout:Generating ssh key... 2024-09-16T08:14:52.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-mon[23701]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:14:52.783 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: Generating public/private rsa key pair. 2024-09-16T08:14:52.783 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: Your identification has been saved in /tmp/tmpum8ikts0/key 2024-09-16T08:14:52.783 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: Your public key has been saved in /tmp/tmpum8ikts0/key.pub 2024-09-16T08:14:52.783 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: The key fingerprint is: 2024-09-16T08:14:52.783 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: SHA256:q88c7GdYqf7VX4xq2pUvsFR1M4Jdsh6WsUqafXPME/I ceph-8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:52.783 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: The key's randomart image is: 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: +---[RSA 3072]----+ 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: | oo.. | 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: | . o*oo| 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: | . B.o+| 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: | = + B .| 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: | So + = E | 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: | . .o +.o+.| 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: | ++ ..o= o| 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: | =o.oooo.o.| 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: | .o*+ooo .o| 2024-09-16T08:14:52.784 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:14:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: +----[SHA256]-----+ 2024-09-16T08:14:53.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:53 smithi090 ceph-mon[23701]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:14:53.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:53 smithi090 ceph-mon[23701]: Generating ssh key... 2024-09-16T08:14:53.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:53 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:14:53.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:53 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:14:54.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:54 smithi090 ceph-mon[23701]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:14:54.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCYPsYLXThl9Nt8iBlGrDsc4jr8cRedFWTQN+c28/iDbUNn4Z4NdHFC9v0dtliiCWvfJcn+BoAaQYpZjA8W//nnkdsSJK3YrF1TYDxDobY5M+LNezxBYrIcWlac3JhRmtNcQFERE3pI86CqwtrkX6Xhkj2tI3qRQ8PHf2sTMFbr9ek82S4XeimzQXI8RjlEud9cYyLgjsHomB14bhcJrw7fliCgrsu3GyFXs4wZoKMl8iKv07/rbADJy2o75XusijHHDjMZjh8T2mJkM2dWJMGh407Kwnkk120llIIPw7xX89Xf0LC6kuUIOw86O8EvH+xdXjPsUGBelNagHN+nM5T9PKDRXLv83t5JcNnqJY3qJxV6hPA93+FjmHt1IrduPbFbtQx0BHx/EY0wJPWpKPdWXS74Cv9+JvUXj1HtdXZYwlrgZ8H/cukt6MQ5YAZDcPSo+H/hxTaLc3SZCJGwM6g50AF+739I8ockIQgAvciWuYYtTNVfJUCUb2Y8n9dBXHk= ceph-8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:14:54.960 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T08:14:54.960 INFO:teuthology.orchestra.run.smithi090.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T08:14:54.967 INFO:teuthology.orchestra.run.smithi090.stdout:Adding host smithi090... 2024-09-16T08:14:56.042 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:55 smithi090 ceph-mon[23701]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:14:56.977 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:56 smithi090 ceph-mon[23701]: Deploying cephadm binary to smithi090 2024-09-16T08:14:58.628 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Added host 'smithi090' with addr '172.21.15.90' 2024-09-16T08:14:58.628 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mon service... 2024-09-16T08:14:59.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:59 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:14:59.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:59 smithi090 ceph-mon[23701]: Added host smithi090 2024-09-16T08:14:59.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:59 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:14:59.535 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:14:59 smithi090 ceph-mon[23701]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:00.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:00 smithi090 ceph-mon[23701]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:15:00.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:00 smithi090 ceph-mon[23701]: Saving service mon spec with placement count:5 2024-09-16T08:15:00.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:00 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:15:00.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:00 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:15:00.553 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T08:15:00.553 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mgr service... 2024-09-16T08:15:01.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:01 smithi090 ceph-mon[23701]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:01.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:01 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:15:01.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:01 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:15:01.671 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T08:15:02.223 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:02 smithi090 ceph-mon[23701]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:15:02.223 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:02 smithi090 ceph-mon[23701]: Saving service mgr spec with placement count:2 2024-09-16T08:15:03.430 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:03 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/350147858' entity='client.admin' 2024-09-16T08:15:03.430 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:03 smithi090 ceph-mon[23701]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:03.430 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:03 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:15:03.430 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:03 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:15:03.430 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:03 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:15:03.430 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:03 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' 2024-09-16T08:15:03.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:03 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:15:03.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:03 smithi090 ceph-mon[23701]: from='mgr.14118 172.21.15.90:0/2812431566' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]': finished 2024-09-16T08:15:04.426 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling the dashboard module... 2024-09-16T08:15:04.469 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:04 smithi090 ceph-mon[23701]: Deploying daemon agent.smithi090 on smithi090 2024-09-16T08:15:04.469 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:04 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/380570165' entity='client.admin' 2024-09-16T08:15:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-mon[23701]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3005962701' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T08:15:05.375 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ignoring --setuser ceph since I am not root 2024-09-16T08:15:05.375 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ignoring --setgroup ceph since I am not root 2024-09-16T08:15:05.375 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:05.374+0000 7f1476c21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T08:15:05.662 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:05.432+0000 7f1476c21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T08:15:05.662 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:05.589+0000 7f1476c21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T08:15:05.662 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:05.661+0000 7f1476c21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T08:15:05.934 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:05.829+0000 7f1476c21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:15:06.222 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:05 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:05.982+0000 7f1476c21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T08:15:06.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:06 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3005962701' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T08:15:06.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:06 smithi090 ceph-mon[23701]: mgrmap e9: y(active, since 26s) 2024-09-16T08:15:06.550 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:06 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:06.304+0000 7f1476c21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T08:15:06.550 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:06 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:06.367+0000 7f1476c21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T08:15:06.811 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:06 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:06.553+0000 7f1476c21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T08:15:07.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:15:07.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T08:15:07.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T08:15:07.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T08:15:07.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T08:15:07.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:15:07.087 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-09-16T08:15:07.087 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 9... 2024-09-16T08:15:07.480 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:07 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/4081079641' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T08:15:07.481 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:07 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:07.444+0000 7f1476c21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T08:15:07.811 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:07 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:07.512+0000 7f1476c21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T08:15:08.468 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:08.213+0000 7f1476c21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T08:15:08.468 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:08.326+0000 7f1476c21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T08:15:08.739 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:08.466+0000 7f1476c21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:15:08.739 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:08.526+0000 7f1476c21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T08:15:08.739 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:08.670+0000 7f1476c21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T08:15:08.739 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:08.737+0000 7f1476c21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T08:15:09.017 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:08.797+0000 7f1476c21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T08:15:09.017 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:08.953+0000 7f1476c21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T08:15:09.017 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:09.017+0000 7f1476c21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T08:15:09.280 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:09.082+0000 7f1476c21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T08:15:09.280 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:09.163+0000 7f1476c21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T08:15:09.280 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:09.222+0000 7f1476c21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T08:15:09.280 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:09.280+0000 7f1476c21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T08:15:10.311 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:10.043+0000 7f1476c21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: Active manager daemon y restarted 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: Activating manager daemon y 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: mgrmap e10: y(active, starting, since 0.0032124s) 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:15:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: Manager daemon y is now available 2024-09-16T08:15:10.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:10.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:10.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:15:10.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:10 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:15:11.716 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:11.378+0000 7f13dd0ab640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:08:15:11] ENGINE Error in HTTPServer.serve 2024-09-16T08:15:11.716 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: Traceback (most recent call last): 2024-09-16T08:15:11.716 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:15:11.716 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self._connections.run(self.expiration_interval) 2024-09-16T08:15:11.716 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self._run(expiration_interval) 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: s = self.context.wrap_socket( 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: return self.sslsocket_class._create( 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:15:11.717 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self.do_handshake() 2024-09-16T08:15:11.718 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:15:11.718 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self._sslobj.do_handshake() 2024-09-16T08:15:11.718 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:15:11.718 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:15:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:15:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:15:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T08:15:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T08:15:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:15:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 9 is available 2024-09-16T08:15:11.719 INFO:teuthology.orchestra.run.smithi090.stdout:Generating a dashboard self-signed certificate... 2024-09-16T08:15:12.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:12 smithi090 ceph-mon[23701]: mgrmap e11: y(active, since 1.0061s) 2024-09-16T08:15:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:12 smithi090 ceph-mon[23701]: [16/Sep/2024:08:15:11] ENGINE Bus STARTING 2024-09-16T08:15:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:12 smithi090 ceph-mon[23701]: [16/Sep/2024:08:15:11] ENGINE Serving on http://172.21.15.90:8765 2024-09-16T08:15:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:15:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:12.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T08:15:12.893 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial admin user... 2024-09-16T08:15:13.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:13 smithi090 ceph-mon[23701]: [16/Sep/2024:08:15:11] ENGINE Serving on https://172.21.15.90:7150 2024-09-16T08:15:13.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:13 smithi090 ceph-mon[23701]: [16/Sep/2024:08:15:11] ENGINE Bus STARTED 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:13 smithi090 ceph-mon[23701]: [16/Sep/2024:08:15:11] ENGINE Error in HTTPServer.serve 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._run(expiration_interval) 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:15:13.101 INFO:journalctl@ceph.mon.a.smithi090.stdout: s = self.context.wrap_socket( 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.sslsocket_class._create( 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout: self.do_handshake() 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._sslobj.do_handshake() 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:13 smithi090 ceph-mon[23701]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:13.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:13 smithi090 ceph-mon[23701]: mgrmap e12: y(active, since 2s) 2024-09-16T08:15:14.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:14 smithi090 ceph-mon[23701]: Deploying daemon agent.smithi090 on smithi090 2024-09-16T08:15:14.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:14 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:14.478 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$xpCnkeMgVfastbyd0uWlduWp4LPwXtSSrOSvGVy.v/q3maQbHygw2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726474513, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T08:15:14.478 INFO:teuthology.orchestra.run.smithi090.stdout:Fetching dashboard port number... 2024-09-16T08:15:15.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:15 smithi090 ceph-mon[23701]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:15:15.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:15 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3831799414' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T08:15:15.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T08:15:15.610 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:15:15.610 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-09-16T08:15:15.617 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:15:15.617 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-16T08:15:15.617 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-09-16T08:15:15.617 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T08:15:15.619 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph Dashboard is now available at: 2024-09-16T08:15:15.620 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:15.620 INFO:teuthology.orchestra.run.smithi090.stdout: URL: https://smithi090.front.sepia.ceph.com:8443/ 2024-09-16T08:15:15.620 INFO:teuthology.orchestra.run.smithi090.stdout: User: admin 2024-09-16T08:15:15.620 INFO:teuthology.orchestra.run.smithi090.stdout: Password: ffx3tssj94 2024-09-16T08:15:15.620 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:15.620 INFO:teuthology.orchestra.run.smithi090.stdout:Saving cluster configuration to /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config directory 2024-09-16T08:15:15.620 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling autotune for osd_memory_target 2024-09-16T08:15:17.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:17 smithi090 ceph-mon[23701]: mgrmap e13: y(active, since 6s) 2024-09-16T08:15:17.919 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T08:15:17.919 INFO:teuthology.orchestra.run.smithi090.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T08:15:17.919 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:17.919 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: ceph telemetry on 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout:For more information see: 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T08:15:17.920 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:17.921 INFO:teuthology.orchestra.run.smithi090.stdout:Bootstrap complete. 2024-09-16T08:15:17.955 INFO:tasks.cephadm:Fetching config... 2024-09-16T08:15:17.956 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:15:17.956 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T08:15:17.973 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T08:15:17.973 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:15:17.973 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T08:15:18.029 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T08:15:18.029 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:15:18.029 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T08:15:18.167 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T08:15:18.167 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:15:18.167 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T08:15:18.223 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T08:15:18.224 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCYPsYLXThl9Nt8iBlGrDsc4jr8cRedFWTQN+c28/iDbUNn4Z4NdHFC9v0dtliiCWvfJcn+BoAaQYpZjA8W//nnkdsSJK3YrF1TYDxDobY5M+LNezxBYrIcWlac3JhRmtNcQFERE3pI86CqwtrkX6Xhkj2tI3qRQ8PHf2sTMFbr9ek82S4XeimzQXI8RjlEud9cYyLgjsHomB14bhcJrw7fliCgrsu3GyFXs4wZoKMl8iKv07/rbADJy2o75XusijHHDjMZjh8T2mJkM2dWJMGh407Kwnkk120llIIPw7xX89Xf0LC6kuUIOw86O8EvH+xdXjPsUGBelNagHN+nM5T9PKDRXLv83t5JcNnqJY3qJxV6hPA93+FjmHt1IrduPbFbtQx0BHx/EY0wJPWpKPdWXS74Cv9+JvUXj1HtdXZYwlrgZ8H/cukt6MQ5YAZDcPSo+H/hxTaLc3SZCJGwM6g50AF+739I8ockIQgAvciWuYYtTNVfJUCUb2Y8n9dBXHk= ceph-8b378aa4-7403-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T08:15:18.365 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCYPsYLXThl9Nt8iBlGrDsc4jr8cRedFWTQN+c28/iDbUNn4Z4NdHFC9v0dtliiCWvfJcn+BoAaQYpZjA8W//nnkdsSJK3YrF1TYDxDobY5M+LNezxBYrIcWlac3JhRmtNcQFERE3pI86CqwtrkX6Xhkj2tI3qRQ8PHf2sTMFbr9ek82S4XeimzQXI8RjlEud9cYyLgjsHomB14bhcJrw7fliCgrsu3GyFXs4wZoKMl8iKv07/rbADJy2o75XusijHHDjMZjh8T2mJkM2dWJMGh407Kwnkk120llIIPw7xX89Xf0LC6kuUIOw86O8EvH+xdXjPsUGBelNagHN+nM5T9PKDRXLv83t5JcNnqJY3qJxV6hPA93+FjmHt1IrduPbFbtQx0BHx/EY0wJPWpKPdWXS74Cv9+JvUXj1HtdXZYwlrgZ8H/cukt6MQ5YAZDcPSo+H/hxTaLc3SZCJGwM6g50AF+739I8ockIQgAvciWuYYtTNVfJUCUb2Y8n9dBXHk= ceph-8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:15:18.378 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCYPsYLXThl9Nt8iBlGrDsc4jr8cRedFWTQN+c28/iDbUNn4Z4NdHFC9v0dtliiCWvfJcn+BoAaQYpZjA8W//nnkdsSJK3YrF1TYDxDobY5M+LNezxBYrIcWlac3JhRmtNcQFERE3pI86CqwtrkX6Xhkj2tI3qRQ8PHf2sTMFbr9ek82S4XeimzQXI8RjlEud9cYyLgjsHomB14bhcJrw7fliCgrsu3GyFXs4wZoKMl8iKv07/rbADJy2o75XusijHHDjMZjh8T2mJkM2dWJMGh407Kwnkk120llIIPw7xX89Xf0LC6kuUIOw86O8EvH+xdXjPsUGBelNagHN+nM5T9PKDRXLv83t5JcNnqJY3qJxV6hPA93+FjmHt1IrduPbFbtQx0BHx/EY0wJPWpKPdWXS74Cv9+JvUXj1HtdXZYwlrgZ8H/cukt6MQ5YAZDcPSo+H/hxTaLc3SZCJGwM6g50AF+739I8ockIQgAvciWuYYtTNVfJUCUb2Y8n9dBXHk= ceph-8b378aa4-7403-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T08:15:18.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:18 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/651192877' entity='client.admin' 2024-09-16T08:15:18.433 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCYPsYLXThl9Nt8iBlGrDsc4jr8cRedFWTQN+c28/iDbUNn4Z4NdHFC9v0dtliiCWvfJcn+BoAaQYpZjA8W//nnkdsSJK3YrF1TYDxDobY5M+LNezxBYrIcWlac3JhRmtNcQFERE3pI86CqwtrkX6Xhkj2tI3qRQ8PHf2sTMFbr9ek82S4XeimzQXI8RjlEud9cYyLgjsHomB14bhcJrw7fliCgrsu3GyFXs4wZoKMl8iKv07/rbADJy2o75XusijHHDjMZjh8T2mJkM2dWJMGh407Kwnkk120llIIPw7xX89Xf0LC6kuUIOw86O8EvH+xdXjPsUGBelNagHN+nM5T9PKDRXLv83t5JcNnqJY3qJxV6hPA93+FjmHt1IrduPbFbtQx0BHx/EY0wJPWpKPdWXS74Cv9+JvUXj1HtdXZYwlrgZ8H/cukt6MQ5YAZDcPSo+H/hxTaLc3SZCJGwM6g50AF+739I8ockIQgAvciWuYYtTNVfJUCUb2Y8n9dBXHk= ceph-8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:15:18.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T08:15:18.693 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:15:19.697 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:19.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:21.059 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T08:15:21.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T08:15:21.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:21 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:21.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:21 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2108527553' entity='client.admin' 2024-09-16T08:15:21.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:21 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:21.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:21 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:21.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:21 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:22.403 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:15:24.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:24.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:24.997 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-09-16T08:15:24.997 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T08:15:24.997 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-09-16T08:15:25.017 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T08:15:25.017 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:15:25.075 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-09-16T08:15:25.076 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch host add smithi193 2024-09-16T08:15:25.454 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:15:25.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:25 smithi090 ceph-mon[23701]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:15:25.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:25.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:25.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:25.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:25.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:25.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:26.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:26 smithi090 ceph-mon[23701]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:15:26.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:26 smithi090 ceph-mon[23701]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:26.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:26 smithi090 ceph-mon[23701]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:15:26.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:26 smithi090 ceph-mon[23701]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:15:26.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:26 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:26.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:26 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:27.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:27 smithi090 ceph-mon[23701]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:15:28.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:28 smithi090 ceph-mon[23701]: Deploying cephadm binary to smithi193 2024-09-16T08:15:29.686 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-09-16T08:15:30.293 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T08:15:30.485 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:15:30.917 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: Added host smithi193 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:30.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:30 smithi090 ceph-mon[23701]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:15:32.122 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:15:32.122 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-09-16T08:15:32.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:31 smithi090 ceph-mon[23701]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:32.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:31 smithi090 ceph-mon[23701]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:15:32.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:31 smithi090 ceph-mon[23701]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:15:32.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:31 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:32.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:31 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:32.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:31 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:15:32.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:31 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]': finished 2024-09-16T08:15:32.728 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T08:15:32.728 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T08:15:32.916 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: Deploying daemon agent.smithi193 on smithi193 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:33.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:33.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:33.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:33.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:33.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:33.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:33.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:33.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:35.107 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-09-16T08:15:35.406 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:35 smithi090 ceph-mon[23701]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:35.406 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:35 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3501131890' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T08:15:35.406 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:35 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:35.684 INFO:tasks.cephadm:Adding mon.a on smithi090 2024-09-16T08:15:35.684 INFO:tasks.cephadm:Adding mon.c on smithi090 2024-09-16T08:15:35.684 INFO:tasks.cephadm:Adding mon.b on smithi193 2024-09-16T08:15:35.684 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi090:172.21.15.90=a;smithi090:[v2:172.21.15.90:3301,v1:172.21.15.90:6790]=c;smithi193:172.21.15.193=b' 2024-09-16T08:15:35.869 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:36.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:36 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3501131890' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T08:15:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:36 smithi090 ceph-mon[23701]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:15:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:36 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:36 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:37.386 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-09-16T08:15:37.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:37 smithi090 ceph-mon[23701]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:37.859 DEBUG:teuthology.orchestra.run.smithi090:mon.c> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T08:15:37.862 DEBUG:teuthology.orchestra.run.smithi193:mon.b> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T08:15:37.865 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T08:15:37.865 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:15:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi090:[v2:172.21.15.90:3301,v1:172.21.15.90:6790]=c;smithi193:172.21.15.193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:15:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi090:[v2:172.21.15.90:3301,v1:172.21.15.90:6790]=c;smithi193:172.21.15.193=b;count:3 2024-09-16T08:15:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:38.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:38.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:38.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:38.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: Deploying daemon mon.b on smithi193 2024-09-16T08:15:38.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:38 smithi090 ceph-mon[23701]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:38.635 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:41.267 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:41 smithi193 systemd[1]: Starting Ceph mon.b for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:15:41.499 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:41 smithi090 ceph-mon[23701]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:41.846 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:41 smithi193 podman[25560]: 2024-09-16 08:15:41.650709505 +0000 UTC m=+0.069383645 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:15:41.846 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:41 smithi193 podman[25560]: 2024-09-16 08:15:41.845996861 +0000 UTC m=+0.264670997 container create a45dcb4d779321c56c725bacba8c37f1d03e625e1cf4586341daa2d0af9022c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T08:15:41.979 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:15:41.979 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"8b378aa4-7403-11ef-bceb-c7b262605968","modified":"2024-09-16T08:14:14.369004Z","created":"2024-09-16T08:14:14.369004Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:15:41.980 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-09-16T08:15:42.101 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 podman[25560]: 2024-09-16 08:15:42.048546596 +0000 UTC m=+0.467220727 container init a45dcb4d779321c56c725bacba8c37f1d03e625e1cf4586341daa2d0af9022c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:15:42.101 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 podman[25560]: 2024-09-16 08:15:42.052496075 +0000 UTC m=+0.471170206 container start a45dcb4d779321c56c725bacba8c37f1d03e625e1cf4586341daa2d0af9022c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=reef-ae7f373, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:15:42.101 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T08:15:42.101 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable), process ceph-mon, pid 2 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: pidfile_write: ignore empty --pid-file 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: load: jerasure load: lrc 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: RocksDB version: 7.9.2 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Git sha 0 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Compile date 2024-09-15 20:36:04 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: DB SUMMARY 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: DB Session ID: PB88XXV0UI28SC0TP5QC 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: CURRENT file: CURRENT 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-16T08:15:42.102 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.error_if_exists: 0 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.create_if_missing: 0 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.paranoid_checks: 1 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.env: 0x55a4b525bbc0 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.info_log: 0x55a4b5c62be0 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T08:15:42.103 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.statistics: (nil) 2024-09-16T08:15:42.105 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.use_fsync: 0 2024-09-16T08:15:42.105 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_log_file_size: 0 2024-09-16T08:15:42.105 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T08:15:42.105 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T08:15:42.105 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.allow_fallocate: 1 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.use_direct_reads: 0 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.db_log_dir: 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.wal_dir: 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T08:15:42.106 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.write_buffer_manager: 0x55a4b5c70320 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T08:15:42.107 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.unordered_write: 0 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.row_cache: None 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.wal_filter: None 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T08:15:42.108 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.two_write_queues: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.wal_compression: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.atomic_flush: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.log_readahead_size: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T08:15:42.109 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_background_jobs: 2 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_background_compactions: -1 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_subcompactions: 1 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T08:15:42.110 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_open_files: -1 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_background_flushes: -1 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Compression algorithms supported: 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: kZSTD supported: 0 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: kXpressCompression supported: 0 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: kBZip2Compression supported: 0 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: kLZ4Compression supported: 1 2024-09-16T08:15:42.111 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: kZlibCompression supported: 1 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: kSnappyCompression supported: 1 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.merge_operator: 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_filter: None 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T08:15:42.112 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a4b5c627e0) 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: index_type: 0 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: data_block_index_type: 0 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: index_shortening: 1 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: checksum: 4 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: no_block_cache: 0 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_cache: 0x55a4b5c57350 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_cache_options: 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: capacity : 536870912 2024-09-16T08:15:42.113 INFO:journalctl@ceph.mon.b.smithi193.stdout: num_shard_bits : 4 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: strict_capacity_limit : 0 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_cache_compressed: (nil) 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: persistent_cache: (nil) 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_size: 4096 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_size_deviation: 10 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_restart_interval: 16 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: index_block_restart_interval: 1 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: metadata_block_size: 4096 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: partition_filters: 0 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: use_delta_encoding: 1 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: filter_policy: bloomfilter 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: whole_key_filtering: 1 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: verify_compression: 0 2024-09-16T08:15:42.114 INFO:journalctl@ceph.mon.b.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout: format_version: 5 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout: enable_index_compression: 1 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout: block_align: 0 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout: max_auto_readahead_size: 262144 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout: prepopulate_block_cache: 0 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout: initial_auto_readahead_size: 8192 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression: NoCompression 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.num_levels: 7 2024-09-16T08:15:42.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T08:15:42.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T08:15:42.117 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T08:15:42.118 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.inplace_update_support: 0 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T08:15:42.119 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.bloom_locality: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.max_successive_merges: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.ttl: 2592000 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T08:15:42.120 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.enable_blob_files: false 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.min_blob_size: 0 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a7129230-57c0-4f40-9899-0bb3c8ebf7a2 2024-09-16T08:15:42.121 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726474542080586, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726474542081001, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726474542, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a7129230-57c0-4f40-9899-0bb3c8ebf7a2", "db_session_id": "PB88XXV0UI28SC0TP5QC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726474542081098, "job": 1, "event": "recovery_finished"} 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a4b5c82e00 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: DB pointer 0x55a4b5d76000 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout: ** DB Stats ** 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:15:42.122 INFO:journalctl@ceph.mon.b.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: ** Compaction Stats [default] ** 2024-09-16T08:15:42.123 INFO:journalctl@ceph.mon.b.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:15:42.124 INFO:journalctl@ceph.mon.b.smithi193.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout: Block cache BinnedLRUCache@0x55a4b5c57350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: starting mon.b rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(???) e0 preinit fsid 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).mds e1 new map 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout: e1 2024-09-16T08:15:42.125 INFO:journalctl@ceph.mon.b.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout: legacy client fscid: -1 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout: 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout: No filesystems configured 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:15:42.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]': finished 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: Deploying daemon agent.smithi193 on smithi193 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:42.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3501131890' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3501131890' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi090:[v2:172.21.15.90:3301,v1:172.21.15.90:6790]=c;smithi193:172.21.15.193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi090:[v2:172.21.15.90:3301,v1:172.21.15.90:6790]=c;smithi193:172.21.15.193=b;count:3 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:42.129 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:42.130 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:42.130 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: Deploying daemon mon.b on smithi193 2024-09-16T08:15:42.130 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:42.130 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:42.130 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 ceph-mon[25614]: mon.b@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-09-16T08:15:42.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 bash[25560]: a45dcb4d779321c56c725bacba8c37f1d03e625e1cf4586341daa2d0af9022c5 2024-09-16T08:15:42.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:42 smithi193 systemd[1]: Started Ceph mon.b for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:15:43.707 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T08:15:43.708 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:15:43.922 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:15:47.500 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:47.500 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:15:47.500 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: mon.a calling monitor election 2024-09-16T08:15:47.500 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.500 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: mon.b calling monitor election 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: monmap e2: 2 mons at {a=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],b=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: fsmap 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:15:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: mgrmap e13: y(active, since 37s) 2024-09-16T08:15:47.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: overall HEALTH_OK 2024-09-16T08:15:47.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:47.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:47.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:47.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:47.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: mon.a calling monitor election 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: mon.b calling monitor election 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: monmap e2: 2 mons at {a=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],b=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: fsmap 2024-09-16T08:15:47.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:15:47.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: mgrmap e13: y(active, since 37s) 2024-09-16T08:15:47.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: overall HEALTH_OK 2024-09-16T08:15:47.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:47.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:47.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:47.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:47.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:48.484 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:48 smithi090 ceph-mon[23701]: Deploying daemon mon.c on smithi090 2024-09-16T08:15:48.484 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:48.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:48 smithi193 ceph-mon[25614]: Deploying daemon mon.c on smithi090 2024-09-16T08:15:48.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:48.508 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:15:48.508 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"8b378aa4-7403-11ef-bceb-c7b262605968","modified":"2024-09-16T08:15:42.106694Z","created":"2024-09-16T08:14:14.369004Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T08:15:48.508 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-09-16T08:15:49.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:49 smithi090 ceph-mon[23701]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:49.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:49 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/3733502061' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:15:49.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:49 smithi193 ceph-mon[25614]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:49.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:49 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/3733502061' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:15:50.143 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T08:15:50.143 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:15:50.311 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:50 smithi090 systemd[1]: Started Ceph mon.c for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:15:50.342 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:15:55.352 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:55.352 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:15:55.352 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: mon.a calling monitor election 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: mon.b calling monitor election 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: mon.c calling monitor election 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T08:15:55.353 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: monmap e3: 3 mons at {a=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],b=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0],c=[v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: fsmap 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: mgrmap e13: y(active, since 45s) 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: overall HEALTH_OK 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.354 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:55 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:55.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:15:55.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: mon.a calling monitor election 2024-09-16T08:15:55.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: mon.b calling monitor election 2024-09-16T08:15:55.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:55.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: mon.c calling monitor election 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: monmap e3: 3 mons at {a=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],b=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0],c=[v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: fsmap 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: mgrmap e13: y(active, since 45s) 2024-09-16T08:15:55.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: overall HEALTH_OK 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: mon.a calling monitor election 2024-09-16T08:15:55.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: mon.b calling monitor election 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: mon.c calling monitor election 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T08:15:55.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: monmap e3: 3 mons at {a=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],b=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0],c=[v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: fsmap 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: mgrmap e13: y(active, since 45s) 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: overall HEALTH_OK 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:55 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:55.611 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:15:55.612 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"8b378aa4-7403-11ef-bceb-c7b262605968","modified":"2024-09-16T08:15:50.144439Z","created":"2024-09-16T08:14:14.369004Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3301","nonce":0},{"type":"v1","addr":"172.21.15.90:6790","nonce":0}]},"addr":"172.21.15.90:6790/0","public_addr":"172.21.15.90:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-16T08:15:55.612 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-09-16T08:15:56.162 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T08:15:56.163 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T08:15:56.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/274538121' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:56.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/274538121' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.399 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.400 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.400 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:56.400 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:15:56.400 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:56.400 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/274538121' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:56.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:56.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:15:56.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:56.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:15:57.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:57 smithi193 ceph-mon[25614]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:57.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:57 smithi193 ceph-mon[25614]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:57.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:57 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:15:57.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:57 smithi193 ceph-mon[25614]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T08:15:57.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:57 smithi193 ceph-mon[25614]: Reconfiguring daemon mon.a on smithi090 2024-09-16T08:15:57.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:57 smithi193 ceph-mon[25614]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:57.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:57.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:57.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[23701]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[23701]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:15:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[23701]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T08:15:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[23701]: Reconfiguring daemon mon.a on smithi090 2024-09-16T08:15:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[23701]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[30018]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[30018]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:15:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:15:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[30018]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T08:15:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[30018]: Reconfiguring daemon mon.a on smithi090 2024-09-16T08:15:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[30018]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:15:59.082 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:59.169 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:59.169 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:59 smithi193 ceph-mon[25614]: Reconfiguring mon.c (monmap changed)... 2024-09-16T08:15:59.169 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:59.169 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:15:59.170 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:59.170 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:59 smithi193 ceph-mon[25614]: Reconfiguring daemon mon.c on smithi090 2024-09-16T08:15:59.170 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:15:59 smithi193 ceph-mon[25614]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:59.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:59.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:59.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[23701]: Reconfiguring mon.c (monmap changed)... 2024-09-16T08:15:59.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:59.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:15:59.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:59.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[23701]: Reconfiguring daemon mon.c on smithi090 2024-09-16T08:15:59.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[23701]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:15:59.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:59.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:15:59.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[30018]: Reconfiguring mon.c (monmap changed)... 2024-09-16T08:15:59.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:15:59.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:15:59.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:15:59.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[30018]: Reconfiguring daemon mon.c on smithi090 2024-09-16T08:15:59.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:15:59 smithi090 ceph-mon[30018]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:00.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.460 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.460 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.460 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.460 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.460 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:00.814 INFO:teuthology.orchestra.run.smithi090.stdout:# minimal ceph.conf for 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:16:00.814 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-09-16T08:16:00.814 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:16:00.814 INFO:teuthology.orchestra.run.smithi090.stdout: mon_host = [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] 2024-09-16T08:16:01.311 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:01 smithi090 ceph-mon[30018]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:01.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:01 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3404534890' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:01.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:01 smithi090 ceph-mon[23701]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:01.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:01 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3404534890' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:01.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:01 smithi193 ceph-mon[25614]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:01 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3404534890' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:01.992 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T08:16:01.992 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:16:01.992 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T08:16:02.019 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:16:02.020 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:16:02.085 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T08:16:02.085 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T08:16:02.113 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T08:16:02.113 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:16:02.180 INFO:tasks.cephadm:Adding mgr.y on smithi090 2024-09-16T08:16:02.180 INFO:tasks.cephadm:Adding mgr.x on smithi193 2024-09-16T08:16:02.180 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi090=y;smithi193=x' 2024-09-16T08:16:02.872 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:02 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:02.872 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:02 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:02.872 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:02 smithi193 ceph-mon[25614]: Reconfiguring mon.b (monmap changed)... 2024-09-16T08:16:02.872 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:02 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:16:02.873 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:02 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:16:02.873 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:02 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:02.873 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:02 smithi193 ceph-mon[25614]: Reconfiguring daemon mon.b on smithi193 2024-09-16T08:16:02.873 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:02 smithi193 ceph-mon[25614]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:02.875 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:16:02.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:02.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:02.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[23701]: Reconfiguring mon.b (monmap changed)... 2024-09-16T08:16:02.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:16:02.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:16:02.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:02.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[23701]: Reconfiguring daemon mon.b on smithi193 2024-09-16T08:16:02.992 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[23701]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:02.992 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:02.992 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:02.992 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[30018]: Reconfiguring mon.b (monmap changed)... 2024-09-16T08:16:02.992 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:16:02.992 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:16:02.992 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:02.992 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[30018]: Reconfiguring daemon mon.b on smithi193 2024-09-16T08:16:02.992 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:02 smithi090 ceph-mon[30018]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:04.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.115 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:04.116 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:16:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:04.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:04.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:16:04.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:04.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:04.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:04.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:04.495 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-09-16T08:16:05.032 DEBUG:teuthology.orchestra.run.smithi193:mgr.x> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T08:16:05.035 DEBUG:tasks.cephadm:set 0 configs 2024-09-16T08:16:05.035 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T08:16:05.035 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:16:05.035 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T08:16:05.055 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T08:16:05.055 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-09-16T08:16:05.113 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T08:16:05.113 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:16:05.113 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T08:16:05.113 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:16:05.113 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:16:05.113 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-16 08:15:58.948149325 +0000 2024-09-16T08:16:05.113 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-16 08:13:03.014859988 +0000 2024-09-16T08:16:05.114 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-16 08:13:03.014859988 +0000 2024-09-16T08:16:05.114 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-16 08:13:03.014859988 +0000 2024-09-16T08:16:05.114 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T08:16:05.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:05.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:05.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:05.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:05.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:05.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:05.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:16:05.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T08:16:05.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T08:16:05.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:16:05.182 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:05.182 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-16T08:16:05.182 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-16T08:16:05.182 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000251266 s, 2.0 MB/s 2024-09-16T08:16:05.183 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:16:05.186 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:05.240 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-09-16T08:16:05.296 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T08:16:05.296 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:16:05.296 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-16T08:16:05.296 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:16:05.296 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:16:05.296 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-16 08:15:58.948149325 +0000 2024-09-16T08:16:05.296 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-16 08:13:03.339854740 +0000 2024-09-16T08:16:05.297 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-16 08:13:03.339854740 +0000 2024-09-16T08:16:05.297 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-16 08:13:03.339854740 +0000 2024-09-16T08:16:05.297 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T08:16:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-16T08:16:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-16T08:16:05.361 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000187913 s, 2.7 MB/s 2024-09-16T08:16:05.362 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T08:16:05.419 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-09-16T08:16:05.474 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T08:16:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:16:05.475 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-16T08:16:05.475 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:16:05.475 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:16:05.475 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-16 08:15:58.948149325 +0000 2024-09-16T08:16:05.475 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-16 08:13:03.668849427 +0000 2024-09-16T08:16:05.475 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-16 08:13:03.668849427 +0000 2024-09-16T08:16:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-16 08:13:03.668849427 +0000 2024-09-16T08:16:05.475 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T08:16:05.538 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-16T08:16:05.539 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-16T08:16:05.539 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000217536 s, 2.4 MB/s 2024-09-16T08:16:05.540 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T08:16:05.596 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-09-16T08:16:05.651 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T08:16:05.651 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:16:05.651 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-16T08:16:05.651 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:16:05.651 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:16:05.651 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-16 08:15:58.948149325 +0000 2024-09-16T08:16:05.651 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-16 08:13:03.987844275 +0000 2024-09-16T08:16:05.652 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-16 08:13:03.987844275 +0000 2024-09-16T08:16:05.652 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-16 08:13:03.987844275 +0000 2024-09-16T08:16:05.652 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T08:16:05.714 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-16T08:16:05.714 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-16T08:16:05.714 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.00019271 s, 2.7 MB/s 2024-09-16T08:16:05.715 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T08:16:05.772 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T08:16:05.773 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T08:16:05.789 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T08:16:05.789 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-09-16T08:16:05.844 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T08:16:05.844 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:16:05.845 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T08:16:05.845 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:16:05.845 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:16:05.845 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-16 08:15:58.469809547 +0000 2024-09-16T08:16:05.845 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-16 08:13:03.017767950 +0000 2024-09-16T08:16:05.845 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-16 08:13:03.017767950 +0000 2024-09-16T08:16:05.845 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-16 08:13:03.017767950 +0000 2024-09-16T08:16:05.845 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T08:16:05.909 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-16T08:16:05.909 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-16T08:16:05.909 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000209415 s, 2.4 MB/s 2024-09-16T08:16:05.910 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T08:16:05.969 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-09-16T08:16:06.024 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T08:16:06.024 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:16:06.025 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T08:16:06.025 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:16:06.025 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:16:06.025 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-16 08:15:58.469809547 +0000 2024-09-16T08:16:06.025 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-16 08:13:03.345762403 +0000 2024-09-16T08:16:06.025 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-16 08:13:03.345762403 +0000 2024-09-16T08:16:06.025 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-16 08:13:03.345762403 +0000 2024-09-16T08:16:06.025 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T08:16:06.089 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-16T08:16:06.089 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-16T08:16:06.089 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000204537 s, 2.5 MB/s 2024-09-16T08:16:06.090 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T08:16:06.147 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-09-16T08:16:06.158 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:05 smithi193 ceph-mon[25614]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi090=y;smithi193=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:16:06.158 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:05 smithi193 ceph-mon[25614]: Saving service mgr spec with placement smithi090=y;smithi193=x;count:2 2024-09-16T08:16:06.158 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:05 smithi193 ceph-mon[25614]: Deploying daemon mgr.x on smithi193 2024-09-16T08:16:06.173 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T08:16:06.173 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:16:06.173 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T08:16:06.173 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:16:06.173 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:16:06.173 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-16 08:15:58.469809547 +0000 2024-09-16T08:16:06.173 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-16 08:13:03.680756737 +0000 2024-09-16T08:16:06.173 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-16 08:13:03.680756737 +0000 2024-09-16T08:16:06.173 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-16 08:13:03.680756737 +0000 2024-09-16T08:16:06.174 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T08:16:06.236 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-16T08:16:06.236 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-16T08:16:06.236 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000212836 s, 2.4 MB/s 2024-09-16T08:16:06.237 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T08:16:06.294 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-09-16T08:16:06.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:05 smithi090 ceph-mon[23701]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi090=y;smithi193=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:16:06.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:05 smithi090 ceph-mon[23701]: Saving service mgr spec with placement smithi090=y;smithi193=x;count:2 2024-09-16T08:16:06.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:05 smithi090 ceph-mon[23701]: Deploying daemon mgr.x on smithi193 2024-09-16T08:16:06.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:05 smithi090 ceph-mon[30018]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi090=y;smithi193=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:16:06.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:05 smithi090 ceph-mon[30018]: Saving service mgr spec with placement smithi090=y;smithi193=x;count:2 2024-09-16T08:16:06.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:05 smithi090 ceph-mon[30018]: Deploying daemon mgr.x on smithi193 2024-09-16T08:16:06.350 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T08:16:06.350 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:16:06.350 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-16T08:16:06.350 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:16:06.351 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:16:06.351 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-09-16 08:15:58.469809547 +0000 2024-09-16T08:16:06.351 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-09-16 08:13:04.009751174 +0000 2024-09-16T08:16:06.351 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-09-16 08:13:04.009751174 +0000 2024-09-16T08:16:06.351 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-09-16 08:13:04.009751174 +0000 2024-09-16T08:16:06.351 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T08:16:06.421 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-09-16T08:16:06.421 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-09-16T08:16:06.421 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000264835 s, 1.9 MB/s 2024-09-16T08:16:06.422 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T08:16:06.479 INFO:tasks.cephadm:Deploying osd.0 on smithi090 with /dev/vg_nvme/lv_4... 2024-09-16T08:16:06.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T08:16:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:16:06.753 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:06 smithi193 systemd[1]: Starting Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:16:07.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:06 smithi090 ceph-mon[23701]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:07.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:06 smithi090 ceph-mon[30018]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:07.081 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:06 smithi193 ceph-mon[25614]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:07.081 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:06 smithi193 podman[27812]: 2024-09-16 08:16:06.730268573 +0000 UTC m=+0.015629389 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:16:07.082 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:06 smithi193 podman[27812]: 2024-09-16 08:16:06.844413752 +0000 UTC m=+0.129774561 container create 4b116b0d4a11e5b7b688116b7e1912ade63c7ced52da0be3902a86d87281fbf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T08:16:07.082 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 podman[27812]: 2024-09-16 08:16:07.054981648 +0000 UTC m=+0.340342467 container init 4b116b0d4a11e5b7b688116b7e1912ade63c7ced52da0be3902a86d87281fbf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T08:16:07.082 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 podman[27812]: 2024-09-16 08:16:07.057918 +0000 UTC m=+0.343278809 container start 4b116b0d4a11e5b7b688116b7e1912ade63c7ced52da0be3902a86d87281fbf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373, GIT_BRANCH=HEAD) 2024-09-16T08:16:07.082 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 ceph-mgr[27831]: -- 172.21.15.193:0/3910499658 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556951684d00 con 0x556951658800 2024-09-16T08:16:07.390 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 bash[27812]: 4b116b0d4a11e5b7b688116b7e1912ade63c7ced52da0be3902a86d87281fbf1 2024-09-16T08:16:07.390 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 systemd[1]: Started Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:16:07.390 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:07.177+0000 7fbb31c44040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T08:16:07.391 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:07.236+0000 7fbb31c44040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T08:16:07.391 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:07.390+0000 7fbb31c44040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T08:16:07.739 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:07.456+0000 7fbb31c44040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T08:16:07.739 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:07.600+0000 7fbb31c44040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:16:07.739 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:07.738+0000 7fbb31c44040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T08:16:08.251 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:08.062+0000 7fbb31c44040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:08.126+0000 7fbb31c44040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:16:08.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:16:08.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:08.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:08.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:08.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:08.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:08.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:08.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:16:08.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:16:08.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:08.744 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:08.312+0000 7fbb31c44040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T08:16:08.974 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:16:08.993 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-09-16T08:16:09.291 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:16:09.291 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:09 smithi193 ceph-mon[25614]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T08:16:09.291 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:09 smithi193 ceph-mon[25614]: Reconfiguring daemon mgr.y on smithi090 2024-09-16T08:16:09.291 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:09 smithi193 ceph-mon[25614]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:09.291 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:09 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:09.216+0000 7fbb31c44040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T08:16:09.291 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:09 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:09.290+0000 7fbb31c44040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T08:16:09.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:16:09.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:09 smithi090 ceph-mon[23701]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T08:16:09.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:09 smithi090 ceph-mon[23701]: Reconfiguring daemon mgr.y on smithi090 2024-09-16T08:16:09.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:09 smithi090 ceph-mon[23701]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:09.434 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:16:09.434 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:09 smithi090 ceph-mon[30018]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T08:16:09.434 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:09 smithi090 ceph-mon[30018]: Reconfiguring daemon mgr.y on smithi090 2024-09-16T08:16:09.434 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:09 smithi090 ceph-mon[30018]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:10.053 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:16:10.167 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.048+0000 7fbb31c44040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T08:16:10.167 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.167+0000 7fbb31c44040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T08:16:10.487 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.318+0000 7fbb31c44040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:16:10.487 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.379+0000 7fbb31c44040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T08:16:10.751 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.528+0000 7fbb31c44040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T08:16:10.751 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.596+0000 7fbb31c44040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T08:16:10.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.655+0000 7fbb31c44040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T08:16:11.089 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.820+0000 7fbb31c44040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T08:16:11.089 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.891+0000 7fbb31c44040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T08:16:11.089 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:10.959+0000 7fbb31c44040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T08:16:11.089 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:11.040+0000 7fbb31c44040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:16:11.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.403 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:11.404 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:11.405 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:11.101+0000 7fbb31c44040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T08:16:11.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:11.160+0000 7fbb31c44040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T08:16:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:11.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:11.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:11.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:11.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:16:11.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:11.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:11.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:11.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:11.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:12.251 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:16:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:16:11.931+0000 7fbb31c44040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T08:16:12.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:16:12.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:16:12.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:12.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:12 smithi193 ceph-mon[25614]: Standby manager daemon x started 2024-09-16T08:16:12.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:12 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:16:12.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:12 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:16:12.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:12 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:16:12.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:12 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:16:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:16:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:16:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[23701]: Standby manager daemon x started 2024-09-16T08:16:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:16:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:16:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:16:12.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:16:12.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:16:12.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:16:12.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:12.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[30018]: Standby manager daemon x started 2024-09-16T08:16:12.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:16:12.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:16:12.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:16:12.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:12 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/2071294979' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:16:13.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:13 smithi090 ceph-mon[23701]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:16:13.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:13 smithi090 ceph-mon[23701]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:13.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:13 smithi090 ceph-mon[23701]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-16T08:16:13.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:16:13.151 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:13 smithi090 ceph-mon[30018]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:16:13.151 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:13 smithi090 ceph-mon[30018]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:13.151 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:13 smithi090 ceph-mon[30018]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-16T08:16:13.151 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:13 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:16:13.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:13 smithi193 ceph-mon[25614]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:16:13.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:13 smithi193 ceph-mon[25614]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:13.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:13 smithi193 ceph-mon[25614]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-16T08:16:13.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:13 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:16:14.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/112429897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5e8461-ab0d-447a-9c03-fbf862340931"}]: dispatch 2024-09-16T08:16:14.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5e8461-ab0d-447a-9c03-fbf862340931"}]: dispatch 2024-09-16T08:16:14.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d5e8461-ab0d-447a-9c03-fbf862340931"}]': finished 2024-09-16T08:16:14.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[23701]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T08:16:14.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:14.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/112429897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5e8461-ab0d-447a-9c03-fbf862340931"}]: dispatch 2024-09-16T08:16:14.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5e8461-ab0d-447a-9c03-fbf862340931"}]: dispatch 2024-09-16T08:16:14.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d5e8461-ab0d-447a-9c03-fbf862340931"}]': finished 2024-09-16T08:16:14.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[30018]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T08:16:14.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:14 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:14.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:14 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/112429897' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5e8461-ab0d-447a-9c03-fbf862340931"}]: dispatch 2024-09-16T08:16:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:14 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d5e8461-ab0d-447a-9c03-fbf862340931"}]: dispatch 2024-09-16T08:16:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:14 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d5e8461-ab0d-447a-9c03-fbf862340931"}]': finished 2024-09-16T08:16:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:14 smithi193 ceph-mon[25614]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T08:16:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:14 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:15.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:15 smithi193 ceph-mon[25614]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:15.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:15 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/78810020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:16:15.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:15 smithi090 ceph-mon[23701]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:15.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:15 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/78810020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:16:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:15 smithi090 ceph-mon[30018]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:15 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/78810020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:16:17.221 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:17 smithi090 ceph-mon[23701]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:17.221 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:17 smithi090 ceph-mon[30018]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:17.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:17 smithi193 ceph-mon[25614]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:19.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:19 smithi193 ceph-mon[25614]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:19.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:19 smithi090 ceph-mon[23701]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:19.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:19 smithi090 ceph-mon[30018]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:21.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:21 smithi090 ceph-mon[23701]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:21.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:21 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:21.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:21 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:21.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:21 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:21.384 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:21 smithi090 ceph-mon[30018]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:21.384 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:21 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:21.384 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:21 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:21.384 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:21 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:21.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:21 smithi193 ceph-mon[25614]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:21.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:21 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:21.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:21 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:21.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:21 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:22.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:22 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi090 2024-09-16T08:16:22.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:22 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T08:16:22.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:22 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:22.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:22 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi090 2024-09-16T08:16:22.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:22 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T08:16:22.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:22 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:22.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:22 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi090 2024-09-16T08:16:22.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:22 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T08:16:22.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:22 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:23.187 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:23 smithi090 ceph-mon[23701]: Deploying daemon osd.0 on smithi090 2024-09-16T08:16:23.187 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:23 smithi090 ceph-mon[23701]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:23.188 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:23 smithi090 ceph-mon[30018]: Deploying daemon osd.0 on smithi090 2024-09-16T08:16:23.188 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:23 smithi090 ceph-mon[30018]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:23.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:23 smithi193 ceph-mon[25614]: Deploying daemon osd.0 on smithi090 2024-09-16T08:16:23.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:23 smithi193 ceph-mon[25614]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:25.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:25 smithi090 ceph-mon[30018]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:25.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:25 smithi090 ceph-mon[23701]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:25.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:25 smithi193 ceph-mon[25614]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[23701]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:27.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[30018]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:27.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:27.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:27.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:27.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:27.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:27.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:27.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:27 smithi193 ceph-mon[25614]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:27.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:27.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:27.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:27.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:27.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:27.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:28.909 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 0 on host 'smithi090' 2024-09-16T08:16:29.360 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:29 smithi193 ceph-mon[25614]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:29.360 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:29 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:29.360 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:29 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.360 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:29 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:16:29.361 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:29 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.361 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:29 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.361 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:29 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:29.361 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:29 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:29.361 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:29 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[30018]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[23701]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:29.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:29.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:16:29.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:29.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:29.483 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:29 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:29.506 DEBUG:teuthology.orchestra.run.smithi090:osd.0> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T08:16:29.508 INFO:tasks.cephadm:Deploying osd.1 on smithi090 with /dev/vg_nvme/lv_3... 2024-09-16T08:16:29.508 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T08:16:29.737 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:16:30.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:30 smithi193 ceph-mon[25614]: Adjusting osd_memory_target on smithi090 to 15040M 2024-09-16T08:16:30.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:30 smithi193 ceph-mon[25614]: from='osd.0 [v2:172.21.15.90:6802/1132188243,v1:172.21.15.90:6803/1132188243]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:16:30.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:30 smithi193 ceph-mon[25614]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:16:30.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:30.400 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:30.401 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[23701]: Adjusting osd_memory_target on smithi090 to 15040M 2024-09-16T08:16:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[23701]: from='osd.0 [v2:172.21.15.90:6802/1132188243,v1:172.21.15.90:6803/1132188243]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:16:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[23701]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:16:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:30.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[30018]: Adjusting osd_memory_target on smithi090 to 15040M 2024-09-16T08:16:30.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[30018]: from='osd.0 [v2:172.21.15.90:6802/1132188243,v1:172.21.15.90:6803/1132188243]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:16:30.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[30018]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:16:30.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:30.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:30.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:31.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[23701]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:31.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[23701]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:16:31.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[23701]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T08:16:31.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:31.394 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[23701]: from='osd.0 [v2:172.21.15.90:6802/1132188243,v1:172.21.15.90:6803/1132188243]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:16:31.394 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[23701]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:16:31.394 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[30018]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:31.394 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[30018]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:16:31.394 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[30018]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T08:16:31.394 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:31.394 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[30018]: from='osd.0 [v2:172.21.15.90:6802/1132188243,v1:172.21.15.90:6803/1132188243]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:16:31.394 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-mon[30018]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:16:31.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:31 smithi193 ceph-mon[25614]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:31.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:31 smithi193 ceph-mon[25614]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:16:31.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:31 smithi193 ceph-mon[25614]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T08:16:31.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:31 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:31.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:31 smithi193 ceph-mon[25614]: from='osd.0 [v2:172.21.15.90:6802/1132188243,v1:172.21.15.90:6803/1132188243]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:16:31.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:31 smithi193 ceph-mon[25614]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:16:31.812 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[34285]: 2024-09-16T08:16:31.566+0000 7f7a4a237640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T08:16:31.812 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:31 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[34285]: 2024-09-16T08:16:31.569+0000 7f7a41841640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:16:32.095 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:16:32.119 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-09-16T08:16:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:16:32.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[23701]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:16:32.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[23701]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T08:16:32.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:32.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:32.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[23701]: from='osd.0 ' entity='osd.0' 2024-09-16T08:16:32.428 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[30018]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:16:32.428 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[30018]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T08:16:32.428 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:32.428 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:32.428 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:32 smithi090 ceph-mon[30018]: from='osd.0 ' entity='osd.0' 2024-09-16T08:16:32.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:32 smithi193 ceph-mon[25614]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:16:32.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:32 smithi193 ceph-mon[25614]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T08:16:32.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:32 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:32.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:32 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:32.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:32 smithi193 ceph-mon[25614]: from='osd.0 ' entity='osd.0' 2024-09-16T08:16:33.285 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[30018]: purged_snaps scrub starts 2024-09-16T08:16:33.285 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[30018]: purged_snaps scrub ok 2024-09-16T08:16:33.285 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[30018]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:33.285 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:33.285 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[30018]: osd.0 [v2:172.21.15.90:6802/1132188243,v1:172.21.15.90:6803/1132188243] boot 2024-09-16T08:16:33.286 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[30018]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T08:16:33.286 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:33.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[23701]: purged_snaps scrub starts 2024-09-16T08:16:33.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[23701]: purged_snaps scrub ok 2024-09-16T08:16:33.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[23701]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:33.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:33.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[23701]: osd.0 [v2:172.21.15.90:6802/1132188243,v1:172.21.15.90:6803/1132188243] boot 2024-09-16T08:16:33.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[23701]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T08:16:33.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:33 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:33.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:33 smithi193 ceph-mon[25614]: purged_snaps scrub starts 2024-09-16T08:16:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:33 smithi193 ceph-mon[25614]: purged_snaps scrub ok 2024-09-16T08:16:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:33 smithi193 ceph-mon[25614]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:16:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:33 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:33 smithi193 ceph-mon[25614]: osd.0 [v2:172.21.15.90:6802/1132188243,v1:172.21.15.90:6803/1132188243] boot 2024-09-16T08:16:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:33 smithi193 ceph-mon[25614]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T08:16:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:33 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:34.417 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:34 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:16:34.417 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:34 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:16:34.417 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:34 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:34.417 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:34 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:16:34.417 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:34 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:16:34.417 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:34 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:34.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:34 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:16:34.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:34 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:16:34.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:34 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:35.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:35 smithi090 ceph-mon[23701]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:16:35.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:35 smithi090 ceph-mon[23701]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:35.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:35 smithi090 ceph-mon[23701]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T08:16:35.435 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:35 smithi090 ceph-mon[30018]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:16:35.435 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:35 smithi090 ceph-mon[30018]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:35.435 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:35 smithi090 ceph-mon[30018]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T08:16:35.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:35 smithi193 ceph-mon[25614]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:16:35.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:35 smithi193 ceph-mon[25614]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:35.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:35 smithi193 ceph-mon[25614]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T08:16:36.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:36 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/405712214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18d07a14-8eb1-484a-a16b-6be128901ae3"}]: dispatch 2024-09-16T08:16:36.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:36 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18d07a14-8eb1-484a-a16b-6be128901ae3"}]: dispatch 2024-09-16T08:16:36.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:36 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18d07a14-8eb1-484a-a16b-6be128901ae3"}]': finished 2024-09-16T08:16:36.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:36 smithi193 ceph-mon[25614]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T08:16:36.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:36 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/405712214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18d07a14-8eb1-484a-a16b-6be128901ae3"}]: dispatch 2024-09-16T08:16:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18d07a14-8eb1-484a-a16b-6be128901ae3"}]: dispatch 2024-09-16T08:16:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18d07a14-8eb1-484a-a16b-6be128901ae3"}]': finished 2024-09-16T08:16:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[23701]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T08:16:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/405712214' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18d07a14-8eb1-484a-a16b-6be128901ae3"}]: dispatch 2024-09-16T08:16:36.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18d07a14-8eb1-484a-a16b-6be128901ae3"}]: dispatch 2024-09-16T08:16:36.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18d07a14-8eb1-484a-a16b-6be128901ae3"}]': finished 2024-09-16T08:16:36.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[30018]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T08:16:36.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:36 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:37.462 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:37 smithi090 ceph-mon[30018]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:37.462 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:37 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1759957537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:16:37.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:37 smithi090 ceph-mon[23701]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:37.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:37 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1759957537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:16:37.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:37 smithi193 ceph-mon[25614]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:37.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:37 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1759957537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:16:39.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:39 smithi090 ceph-mon[23701]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:39.369 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:39 smithi090 ceph-mon[30018]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:39.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:39 smithi193 ceph-mon[25614]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:41.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[23701]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:41.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:41.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:41.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:41.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.308 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[30018]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:41.308 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.308 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.308 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:41.308 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.308 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:41.308 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:41.308 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:41 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:41 smithi193 ceph-mon[25614]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:41 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:41 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:41 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:41 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:41 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:41 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:41.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:41 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:42.264 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:42 smithi090 ceph-mon[23701]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:42.264 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:42 smithi090 ceph-mon[23701]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-16T08:16:42.264 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:42 smithi090 ceph-mon[30018]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:42.264 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:42 smithi090 ceph-mon[30018]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-16T08:16:42.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:42 smithi193 ceph-mon[25614]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:42.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:42 smithi193 ceph-mon[25614]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-16T08:16:44.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:44.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T08:16:44.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:44.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: Deploying daemon osd.0 on smithi090 2024-09-16T08:16:44.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:44.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: Deploying daemon osd.0 on smithi090 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:44.455 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:44.455 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.455 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:44.455 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: Deploying daemon osd.0 on smithi090 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:45 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi090 2024-09-16T08:16:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:45 smithi193 ceph-mon[25614]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:45 smithi193 ceph-mon[25614]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-16T08:16:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:45 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:16:45.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:45 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi090 2024-09-16T08:16:45.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:45 smithi090 ceph-mon[23701]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:45.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:45 smithi090 ceph-mon[23701]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-16T08:16:45.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:45 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:16:45.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:45 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi090 2024-09-16T08:16:45.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:45 smithi090 ceph-mon[30018]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:45.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:45 smithi090 ceph-mon[30018]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-16T08:16:45.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:45 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:16:46.174 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:45 smithi090 systemd[1]: Stopping Ceph osd.0 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:16:46.562 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:46 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[34285]: 2024-09-16T08:16:46.173+0000 7f7a4aa38640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:16:46.562 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:46 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[34285]: 2024-09-16T08:16:46.173+0000 7f7a4aa38640 -1 osd.0 10 *** Got signal Terminated *** 2024-09-16T08:16:46.562 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:46 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[34285]: 2024-09-16T08:16:46.173+0000 7f7a4aa38640 -1 osd.0 10 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T08:16:47.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:47 smithi193 ceph-mon[25614]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:47 smithi193 ceph-mon[25614]: osd.0 marked itself down and dead 2024-09-16T08:16:47.544 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:47 smithi090 ceph-mon[23701]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:47.544 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:47 smithi090 ceph-mon[23701]: osd.0 marked itself down and dead 2024-09-16T08:16:47.544 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:47 smithi090 ceph-mon[30018]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:47.544 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:47 smithi090 ceph-mon[30018]: osd.0 marked itself down and dead 2024-09-16T08:16:47.808 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:47 smithi090 podman[37470]: 2024-09-16 08:16:47.544355383 +0000 UTC m=+1.479273041 container died 08d0ea65da415c5ee4e542f172788f53d442635f85021a96204b0651f50da8fe (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=reef-ae7f373, io.buildah.version=1.37.2) 2024-09-16T08:16:48.062 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:47 smithi090 podman[37470]: 2024-09-16 08:16:47.941034804 +0000 UTC m=+1.875952474 container cleanup 08d0ea65da415c5ee4e542f172788f53d442635f85021a96204b0651f50da8fe (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:16:48.062 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:47 smithi090 bash[37470]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0 2024-09-16T08:16:48.476 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:48 smithi090 podman[37485]: 2024-09-16 08:16:48.159264956 +0000 UTC m=+0.613016048 container remove 08d0ea65da415c5ee4e542f172788f53d442635f85021a96204b0651f50da8fe (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T08:16:48.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[23701]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T08:16:48.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[23701]: Health check failed: 1 host (1 osds) down (OSD_HOST_DOWN) 2024-09-16T08:16:48.477 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[23701]: Health check failed: 1 root (1 osds) down (OSD_ROOT_DOWN) 2024-09-16T08:16:48.477 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[23701]: osdmap e11: 2 total, 0 up, 2 in 2024-09-16T08:16:48.477 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:48.477 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[30018]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T08:16:48.478 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[30018]: Health check failed: 1 host (1 osds) down (OSD_HOST_DOWN) 2024-09-16T08:16:48.478 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[30018]: Health check failed: 1 root (1 osds) down (OSD_ROOT_DOWN) 2024-09-16T08:16:48.478 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[30018]: osdmap e11: 2 total, 0 up, 2 in 2024-09-16T08:16:48.478 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:48.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:48 smithi193 ceph-mon[25614]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T08:16:48.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:48 smithi193 ceph-mon[25614]: Health check failed: 1 host (1 osds) down (OSD_HOST_DOWN) 2024-09-16T08:16:48.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:48 smithi193 ceph-mon[25614]: Health check failed: 1 root (1 osds) down (OSD_ROOT_DOWN) 2024-09-16T08:16:48.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:48 smithi193 ceph-mon[25614]: osdmap e11: 2 total, 0 up, 2 in 2024-09-16T08:16:48.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:48.747 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:48 smithi090 podman[37574]: 2024-09-16 08:16:48.476895352 +0000 UTC m=+0.110163750 container create 72edfd7a4e15378b2c6155180c8d7b84de89b1112c1c3fc40175f67bcaa1ab5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T08:16:48.747 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:48 smithi090 podman[37574]: 2024-09-16 08:16:48.382109154 +0000 UTC m=+0.015377569 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:16:48.747 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:48 smithi090 podman[37574]: 2024-09-16 08:16:48.680905272 +0000 UTC m=+0.314173678 container init 72edfd7a4e15378b2c6155180c8d7b84de89b1112c1c3fc40175f67bcaa1ab5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T08:16:48.748 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:48 smithi090 podman[37574]: 2024-09-16 08:16:48.684009419 +0000 UTC m=+0.317277819 container start 72edfd7a4e15378b2c6155180c8d7b84de89b1112c1c3fc40175f67bcaa1ab5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T08:16:48.748 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:48 smithi090 podman[37574]: 2024-09-16 08:16:48.747633432 +0000 UTC m=+0.380901839 container attach 72edfd7a4e15378b2c6155180c8d7b84de89b1112c1c3fc40175f67bcaa1ab5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373) 2024-09-16T08:16:49.062 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:48 smithi090 podman[37574]: 2024-09-16 08:16:48.86835921 +0000 UTC m=+0.501627610 container died 72edfd7a4e15378b2c6155180c8d7b84de89b1112c1c3fc40175f67bcaa1ab5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=) 2024-09-16T08:16:49.409 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:49 smithi090 ceph-mon[23701]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:49.409 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:49 smithi090 ceph-mon[30018]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:49.410 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:49 smithi090 podman[37597]: 2024-09-16 08:16:49.393224926 +0000 UTC m=+0.519322771 container remove 72edfd7a4e15378b2c6155180c8d7b84de89b1112c1c3fc40175f67bcaa1ab5b (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T08:16:49.413 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:49 smithi193 ceph-mon[25614]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:49.803 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:49 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.0.service: Deactivated successfully. 2024-09-16T08:16:49.803 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:49 smithi090 systemd[1]: Stopped Ceph osd.0 for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:16:49.803 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:49 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.0.service: Consumed 1.565s CPU time. 2024-09-16T08:16:49.803 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:49 smithi090 systemd[1]: Starting Ceph osd.0 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:16:50.061 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:49 smithi090 podman[37697]: 2024-09-16 08:16:49.803076604 +0000 UTC m=+0.108072703 container create 3c7a2560624b8cef60465c8f90383ba6daeefa846a7155e4578206186493b7fb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:16:50.062 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:49 smithi090 podman[37697]: 2024-09-16 08:16:49.710925862 +0000 UTC m=+0.015921927 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:16:50.062 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 podman[37697]: 2024-09-16 08:16:50.023653856 +0000 UTC m=+0.328649912 container init 3c7a2560624b8cef60465c8f90383ba6daeefa846a7155e4578206186493b7fb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T08:16:50.062 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 podman[37697]: 2024-09-16 08:16:50.026992336 +0000 UTC m=+0.331988398 container start 3c7a2560624b8cef60465c8f90383ba6daeefa846a7155e4578206186493b7fb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373) 2024-09-16T08:16:50.323 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 podman[37697]: 2024-09-16 08:16:50.115654072 +0000 UTC m=+0.420650142 container attach 3c7a2560624b8cef60465c8f90383ba6daeefa846a7155e4578206186493b7fb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T08:16:50.806 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate[37713]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T08:16:50.806 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 bash[37697]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T08:16:50.806 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate[37713]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T08:16:50.806 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 bash[37697]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T08:16:50.806 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate[37713]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T08:16:50.806 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 bash[37697]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T08:16:50.806 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate[37713]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T08:16:50.807 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 bash[37697]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T08:16:50.807 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate[37713]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-16T08:16:50.807 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 bash[37697]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-16T08:16:50.807 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate[37713]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T08:16:50.807 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 bash[37697]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T08:16:50.807 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate[37713]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-16T08:16:50.807 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 bash[37697]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-16T08:16:50.807 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 podman[37697]: 2024-09-16 08:16:50.371005581 +0000 UTC m=+0.676001638 container died 3c7a2560624b8cef60465c8f90383ba6daeefa846a7155e4578206186493b7fb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T08:16:51.061 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:50 smithi090 podman[37831]: 2024-09-16 08:16:50.919728364 +0000 UTC m=+0.542882746 container remove 3c7a2560624b8cef60465c8f90383ba6daeefa846a7155e4578206186493b7fb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-activate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T08:16:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-mon[23701]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:51.324 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:51.324 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-mon[30018]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:51.325 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:51.325 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:51.325 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:51.325 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:51 smithi090 podman[37891]: 2024-09-16 08:16:51.030910132 +0000 UTC m=+0.015547623 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:16:51.325 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:51 smithi090 podman[37891]: 2024-09-16 08:16:51.12903576 +0000 UTC m=+0.113673254 container create d8ac704cb999ccfbdac7516e6ae4c8e9c4c405e9fc3ae47cd0b8eca4abe81f55 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:16:51.325 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:51 smithi090 podman[37891]: 2024-09-16 08:16:51.324465386 +0000 UTC m=+0.309102886 container init d8ac704cb999ccfbdac7516e6ae4c8e9c4c405e9fc3ae47cd0b8eca4abe81f55 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:16:51.334 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:51 smithi193 ceph-mon[25614]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:51.334 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:51.334 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:51.334 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:51.663 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:51 smithi090 podman[37891]: 2024-09-16 08:16:51.329409508 +0000 UTC m=+0.314047000 container start d8ac704cb999ccfbdac7516e6ae4c8e9c4c405e9fc3ae47cd0b8eca4abe81f55 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:16:51.663 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-osd[37911]: -- 172.21.15.90:0/1368480261 <== mon.2 v2:172.21.15.90:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e5d77951e0 con 0x55e5d8514400 2024-09-16T08:16:51.663 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:51 smithi090 bash[37891]: d8ac704cb999ccfbdac7516e6ae4c8e9c4c405e9fc3ae47cd0b8eca4abe81f55 2024-09-16T08:16:51.663 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:51 smithi090 systemd[1]: Started Ceph osd.0 for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:16:51.951 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[37906]: 2024-09-16T08:16:51.881+0000 7f4df5a2e640 -1 Falling back to public interface 2024-09-16T08:16:52.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:52.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:52.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:52.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:52.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:52.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T08:16:52.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:52.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:52.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:52.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:52.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:52.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:52.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:52.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T08:16:52.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:52.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:52 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:52.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:52 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:52 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:52 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:52 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:52 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:52 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T08:16:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:52 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:52 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:53.367 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:53 smithi090 ceph-mon[30018]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:53.367 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:53 smithi090 ceph-mon[30018]: Deploying daemon osd.1 on smithi090 2024-09-16T08:16:53.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:53 smithi090 ceph-mon[23701]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:53.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:53 smithi090 ceph-mon[23701]: Deploying daemon osd.1 on smithi090 2024-09-16T08:16:53.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:53 smithi193 ceph-mon[25614]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:53.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:53 smithi193 ceph-mon[25614]: Deploying daemon osd.1 on smithi090 2024-09-16T08:16:54.726 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[37906]: 2024-09-16T08:16:54.604+0000 7f4df5a2e640 -1 osd.0 10 log_to_monitors true 2024-09-16T08:16:55.452 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:55 smithi090 ceph-mon[23701]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:55.498 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:55 smithi090 ceph-mon[23701]: from='osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:16:55.499 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:55 smithi090 ceph-mon[30018]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:55.499 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:55 smithi090 ceph-mon[30018]: from='osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:16:55.499 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:16:55 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[37906]: 2024-09-16T08:16:55.086+0000 7f4ded7d8640 -1 osd.0 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:16:55.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:55 smithi193 ceph-mon[25614]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:55.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:55 smithi193 ceph-mon[25614]: from='osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:16:56.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:56 smithi090 ceph-mon[23701]: from='osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:16:56.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:56 smithi090 ceph-mon[23701]: osdmap e12: 2 total, 0 up, 2 in 2024-09-16T08:16:56.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:56.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:56 smithi090 ceph-mon[23701]: from='osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:16:56.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:56 smithi090 ceph-mon[30018]: from='osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:16:56.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:56 smithi090 ceph-mon[30018]: osdmap e12: 2 total, 0 up, 2 in 2024-09-16T08:16:56.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:56.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:56 smithi090 ceph-mon[30018]: from='osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:16:56.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:56 smithi193 ceph-mon[25614]: from='osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:16:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:56 smithi193 ceph-mon[25614]: osdmap e12: 2 total, 0 up, 2 in 2024-09-16T08:16:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:56.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:56 smithi193 ceph-mon[25614]: from='osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:16:57.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: pgmap v56: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:57.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T08:16:57.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: Health check cleared: OSD_HOST_DOWN (was: 1 host (1 osds) down) 2024-09-16T08:16:57.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: Health check cleared: OSD_ROOT_DOWN (was: 1 root (1 osds) down) 2024-09-16T08:16:57.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:16:57.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494] boot 2024-09-16T08:16:57.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: osdmap e13: 2 total, 1 up, 2 in 2024-09-16T08:16:57.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: pgmap v56: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: Health check cleared: OSD_HOST_DOWN (was: 1 host (1 osds) down) 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: Health check cleared: OSD_ROOT_DOWN (was: 1 root (1 osds) down) 2024-09-16T08:16:57.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494] boot 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: osdmap e13: 2 total, 1 up, 2 in 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:57.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:57.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: pgmap v56: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:57.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: Health check cleared: OSD_HOST_DOWN (was: 1 host (1 osds) down) 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: Health check cleared: OSD_ROOT_DOWN (was: 1 root (1 osds) down) 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: osd.0 [v2:172.21.15.90:6802/2713629494,v1:172.21.15.90:6803/2713629494] boot 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: osdmap e13: 2 total, 1 up, 2 in 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:57.404 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:58.822 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 0,1 on host 'smithi090' 2024-09-16T08:16:59.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: pgmap v58: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:59.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:59.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:16:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:16:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.355 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: pgmap v58: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:59.355 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:59.356 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.356 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:16:59.356 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:16:59.356 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.356 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.356 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:59.356 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:59.356 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:16:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: pgmap v58: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:16:59.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:16:59.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:16:59.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:16:59.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:16:59.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:16:59.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:16:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:16:59.605 DEBUG:teuthology.orchestra.run.smithi090:osd.1> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T08:16:59.607 INFO:tasks.cephadm:Deploying osd.2 on smithi090 with /dev/vg_nvme/lv_2... 2024-09-16T08:16:59.607 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T08:17:00.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:00 smithi090 ceph-mon[30018]: Adjusting osd_memory_target on smithi090 to 7520M 2024-09-16T08:17:00.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:00 smithi090 ceph-mon[30018]: from='osd.1 [v2:172.21.15.90:6810/3883828403,v1:172.21.15.90:6811/3883828403]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T08:17:00.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:00 smithi090 ceph-mon[30018]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T08:17:00.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:00 smithi090 ceph-mon[23701]: Adjusting osd_memory_target on smithi090 to 7520M 2024-09-16T08:17:00.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:00 smithi090 ceph-mon[23701]: from='osd.1 [v2:172.21.15.90:6810/3883828403,v1:172.21.15.90:6811/3883828403]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T08:17:00.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:00 smithi090 ceph-mon[23701]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T08:17:00.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:00 smithi193 ceph-mon[25614]: Adjusting osd_memory_target on smithi090 to 7520M 2024-09-16T08:17:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:00 smithi193 ceph-mon[25614]: from='osd.1 [v2:172.21.15.90:6810/3883828403,v1:172.21.15.90:6811/3883828403]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T08:17:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:00 smithi193 ceph-mon[25614]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T08:17:01.354 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[30018]: pgmap v59: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[30018]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[30018]: osdmap e14: 2 total, 1 up, 2 in 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[30018]: from='osd.1 [v2:172.21.15.90:6810/3883828403,v1:172.21.15.90:6811/3883828403]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[30018]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[23701]: pgmap v59: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[23701]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[23701]: osdmap e14: 2 total, 1 up, 2 in 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[23701]: from='osd.1 [v2:172.21.15.90:6810/3883828403,v1:172.21.15.90:6811/3883828403]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-mon[23701]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:01.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:01 smithi193 ceph-mon[25614]: pgmap v59: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:17:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:01 smithi193 ceph-mon[25614]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T08:17:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:01 smithi193 ceph-mon[25614]: osdmap e14: 2 total, 1 up, 2 in 2024-09-16T08:17:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:01 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:01 smithi193 ceph-mon[25614]: from='osd.1 [v2:172.21.15.90:6810/3883828403,v1:172.21.15.90:6811/3883828403]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:01 smithi193 ceph-mon[25614]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:01.670 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1[38940]: 2024-09-16T08:17:01.523+0000 7f1adaeae640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T08:17:01.671 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1[38940]: 2024-09-16T08:17:01.526+0000 7f1ad2ccf640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:17:01.855 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:17:02.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[23701]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:17:02.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[23701]: osdmap e15: 2 total, 1 up, 2 in 2024-09-16T08:17:02.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:02.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:02.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[23701]: from='osd.1 ' entity='osd.1' 2024-09-16T08:17:02.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[30018]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:17:02.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[30018]: osdmap e15: 2 total, 1 up, 2 in 2024-09-16T08:17:02.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:02.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:02.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:02 smithi090 ceph-mon[30018]: from='osd.1 ' entity='osd.1' 2024-09-16T08:17:02.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:02 smithi193 ceph-mon[25614]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:17:02.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:02 smithi193 ceph-mon[25614]: osdmap e15: 2 total, 1 up, 2 in 2024-09-16T08:17:02.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:02 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:02.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:02 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:02.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:02 smithi193 ceph-mon[25614]: from='osd.1 ' entity='osd.1' 2024-09-16T08:17:03.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[23701]: purged_snaps scrub starts 2024-09-16T08:17:03.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[23701]: purged_snaps scrub ok 2024-09-16T08:17:03.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[23701]: pgmap v62: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[23701]: osd.1 [v2:172.21.15.90:6810/3883828403,v1:172.21.15.90:6811/3883828403] boot 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[23701]: osdmap e16: 2 total, 2 up, 2 in 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[30018]: purged_snaps scrub starts 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[30018]: purged_snaps scrub ok 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[30018]: pgmap v62: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[30018]: osd.1 [v2:172.21.15.90:6810/3883828403,v1:172.21.15.90:6811/3883828403] boot 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[30018]: osdmap e16: 2 total, 2 up, 2 in 2024-09-16T08:17:03.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:03.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:03 smithi193 ceph-mon[25614]: purged_snaps scrub starts 2024-09-16T08:17:03.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:03 smithi193 ceph-mon[25614]: purged_snaps scrub ok 2024-09-16T08:17:03.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:03 smithi193 ceph-mon[25614]: pgmap v62: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:17:03.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:03.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:03 smithi193 ceph-mon[25614]: osd.1 [v2:172.21.15.90:6810/3883828403,v1:172.21.15.90:6811/3883828403] boot 2024-09-16T08:17:03.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:03 smithi193 ceph-mon[25614]: osdmap e16: 2 total, 2 up, 2 in 2024-09-16T08:17:03.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:17:04.651 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[23701]: osdmap e17: 2 total, 2 up, 2 in 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:04.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:04.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:04.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:04.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:04 smithi090 ceph-mon[30018]: osdmap e17: 2 total, 2 up, 2 in 2024-09-16T08:17:04.672 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-09-16T08:17:04.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:04.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:04.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:04.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:04.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:04.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:04 smithi193 ceph-mon[25614]: osdmap e17: 2 total, 2 up, 2 in 2024-09-16T08:17:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:17:07.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:07 smithi090 ceph-mon[23701]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:07.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:07 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:17:07.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:07 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:17:07.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:07 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:07.158 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:07 smithi090 ceph-mon[30018]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:07.158 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:07 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:17:07.158 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:07 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:17:07.158 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:07 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:07.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:07 smithi193 ceph-mon[25614]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:07.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:07 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:17:07.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:07 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:17:07.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:07 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:08.199 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:08 smithi090 ceph-mon[30018]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:17:08.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:08 smithi090 ceph-mon[23701]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:17:08.447 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:08 smithi193 ceph-mon[25614]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:17:09.147 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:09 smithi193 ceph-mon[25614]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:09.147 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:09 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/667891766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55d2c1c2-391b-404c-aa8e-4955fc69038f"}]: dispatch 2024-09-16T08:17:09.147 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:09 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/667891766' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55d2c1c2-391b-404c-aa8e-4955fc69038f"}]': finished 2024-09-16T08:17:09.148 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:09 smithi193 ceph-mon[25614]: osdmap e18: 3 total, 2 up, 3 in 2024-09-16T08:17:09.148 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:09 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:09.238 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[30018]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:09.239 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/667891766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55d2c1c2-391b-404c-aa8e-4955fc69038f"}]: dispatch 2024-09-16T08:17:09.239 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/667891766' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55d2c1c2-391b-404c-aa8e-4955fc69038f"}]': finished 2024-09-16T08:17:09.239 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[30018]: osdmap e18: 3 total, 2 up, 3 in 2024-09-16T08:17:09.239 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:09.239 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[23701]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:09.239 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/667891766' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55d2c1c2-391b-404c-aa8e-4955fc69038f"}]: dispatch 2024-09-16T08:17:09.239 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/667891766' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55d2c1c2-391b-404c-aa8e-4955fc69038f"}]': finished 2024-09-16T08:17:09.239 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[23701]: osdmap e18: 3 total, 2 up, 3 in 2024-09-16T08:17:09.239 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:09 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:10.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:10 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1103451779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:17:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:10 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1103451779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:17:10.445 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:10 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1103451779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:17:11.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:11 smithi090 ceph-mon[30018]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:11.455 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:11.455 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:11.455 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:11.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:11 smithi090 ceph-mon[23701]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:11.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:11.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:11.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:11.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:11 smithi193 ceph-mon[25614]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:13.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:13 smithi193 ceph-mon[25614]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:13.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:13 smithi090 ceph-mon[23701]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:13.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:13 smithi090 ceph-mon[30018]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:15.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:15 smithi193 ceph-mon[25614]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:15.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:15 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T08:17:15.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:15 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:15 smithi090 ceph-mon[23701]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:15 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T08:17:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:15 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:15 smithi090 ceph-mon[30018]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:15 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T08:17:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:15 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:16.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:16 smithi090 ceph-mon[23701]: Deploying daemon osd.2 on smithi090 2024-09-16T08:17:16.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:16 smithi090 ceph-mon[30018]: Deploying daemon osd.2 on smithi090 2024-09-16T08:17:16.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:16 smithi193 ceph-mon[25614]: Deploying daemon osd.2 on smithi090 2024-09-16T08:17:17.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:17 smithi193 ceph-mon[25614]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:17.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:17 smithi090 ceph-mon[23701]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:17.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:17 smithi090 ceph-mon[30018]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:19 smithi193 ceph-mon[25614]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:19.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:19 smithi090 ceph-mon[23701]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:19.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:19 smithi090 ceph-mon[30018]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:20.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:20.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:20.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:20.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:20.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:20.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:20.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:20.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:20.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:20.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:20.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:20.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:21.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:21 smithi193 ceph-mon[25614]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:21.547 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:21 smithi090 ceph-mon[30018]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:21.547 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:21 smithi090 ceph-mon[23701]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:23.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:23 smithi090 ceph-mon[23701]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:23.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:23 smithi090 ceph-mon[23701]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T08:17:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:23 smithi090 ceph-mon[30018]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:23 smithi090 ceph-mon[30018]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T08:17:23.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:23 smithi193 ceph-mon[25614]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:23.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:23 smithi193 ceph-mon[25614]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T08:17:23.907 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 2 on host 'smithi090' 2024-09-16T08:17:24.484 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: osdmap e19: 3 total, 2 up, 3 in 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:24.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: osdmap e20: 3 total, 2 up, 3 in 2024-09-16T08:17:24.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: osdmap e19: 3 total, 2 up, 3 in 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:24.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: osdmap e20: 3 total, 2 up, 3 in 2024-09-16T08:17:24.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T08:17:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: osdmap e19: 3 total, 2 up, 3 in 2024-09-16T08:17:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:24.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:17:24.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: osdmap e20: 3 total, 2 up, 3 in 2024-09-16T08:17:24.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:25.101 DEBUG:teuthology.orchestra.run.smithi090:osd.2> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T08:17:25.104 INFO:tasks.cephadm:Deploying osd.3 on smithi090 with /dev/vg_nvme/lv_1... 2024-09-16T08:17:25.104 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T08:17:25.360 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:17:25.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:25 smithi090 ceph-mon[30018]: Adjusting osd_memory_target on smithi090 to 5013M 2024-09-16T08:17:25.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:25 smithi090 ceph-mon[30018]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:25.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:25 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi090 2024-09-16T08:17:25.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:25 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:25.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:25 smithi090 ceph-mon[23701]: Adjusting osd_memory_target on smithi090 to 5013M 2024-09-16T08:17:25.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:25 smithi090 ceph-mon[23701]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:25.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:25 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi090 2024-09-16T08:17:25.475 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:25.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:25 smithi193 ceph-mon[25614]: Adjusting osd_memory_target on smithi090 to 5013M 2024-09-16T08:17:25.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:25 smithi193 ceph-mon[25614]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:25 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi090 2024-09-16T08:17:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:25 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:26.643 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[30018]: purged_snaps scrub starts 2024-09-16T08:17:26.643 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[30018]: purged_snaps scrub ok 2024-09-16T08:17:26.643 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:26.644 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[30018]: osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361] boot 2024-09-16T08:17:26.644 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[30018]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T08:17:26.644 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:26.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[23701]: purged_snaps scrub starts 2024-09-16T08:17:26.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[23701]: purged_snaps scrub ok 2024-09-16T08:17:26.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:26.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[23701]: osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361] boot 2024-09-16T08:17:26.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[23701]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T08:17:26.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:26 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:26.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:26 smithi193 ceph-mon[25614]: purged_snaps scrub starts 2024-09-16T08:17:26.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:26 smithi193 ceph-mon[25614]: purged_snaps scrub ok 2024-09-16T08:17:26.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:26 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:26.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:26 smithi193 ceph-mon[25614]: osd.2 [v2:172.21.15.90:6818/1303698361,v1:172.21.15.90:6819/1303698361] boot 2024-09-16T08:17:26.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:26 smithi193 ceph-mon[25614]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T08:17:26.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:26 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:17:27.653 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:27.653 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:17:27.653 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T08:17:27.653 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T08:17:27.653 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.654 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:17:27.654 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.654 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:27.654 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.654 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:27.654 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:27.654 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.692 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:27.692 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:17:27.692 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T08:17:27.692 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T08:17:27.692 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.692 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:17:27.692 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.692 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:17:27.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:27.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:27.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:27.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:27.695 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:17:27.713 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-09-16T08:17:27.913 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:17:28.036 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:17:27 smithi090 sudo[44005]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:17:28.036 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:17:27 smithi090 sudo[44005]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:28.036 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:17:27 smithi090 sudo[44005]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:28.036 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:17:27 smithi090 sudo[44005]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:28.312 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44033]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:17:28.312 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44033]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:28.312 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44033]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:28.312 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44033]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:28.704 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:28 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:17:28.704 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:28 smithi090 ceph-mon[30018]: osdmap e23: 3 total, 3 up, 3 in 2024-09-16T08:17:28.704 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:28 smithi090 ceph-mon[30018]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:28.704 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:28 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:28.705 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44050]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:17:28.705 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44050]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:28.705 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44050]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:28.705 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44050]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:28.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:28 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:17:28.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:28 smithi090 ceph-mon[23701]: osdmap e23: 3 total, 3 up, 3 in 2024-09-16T08:17:28.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:28 smithi090 ceph-mon[23701]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:28.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:28 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:28.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:28 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:17:28.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:28 smithi193 ceph-mon[25614]: osdmap e23: 3 total, 3 up, 3 in 2024-09-16T08:17:28.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:28 smithi193 ceph-mon[25614]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:28.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:28 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:28.968 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44054]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:17:28.968 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44054]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:28.968 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44054]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:28.969 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44054]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:28.969 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44059]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:17:28.969 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44059]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:28.969 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44059]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:28.969 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:17:28 smithi090 sudo[44059]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:29.219 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:17:29.219 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:29.219 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44082]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:29.219 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44082]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:29.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44111]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:17:29.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44111]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:29.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44111]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:29.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44111]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:29.998 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:29 smithi193 ceph-mon[25614]: osdmap e24: 3 total, 3 up, 3 in 2024-09-16T08:17:29.998 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:29 smithi193 ceph-mon[25614]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T08:17:29.998 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:29 smithi193 ceph-mon[25614]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:17:29.999 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:29 smithi193 sudo[29364]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:17:29.999 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:29 smithi193 sudo[29364]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:29.999 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:29 smithi193 sudo[29364]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:29.999 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:29 smithi193 sudo[29364]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:30.061 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:29 smithi090 ceph-mon[30018]: osdmap e24: 3 total, 3 up, 3 in 2024-09-16T08:17:30.061 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:29 smithi090 ceph-mon[30018]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T08:17:30.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:29 smithi090 ceph-mon[30018]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:17:30.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44115]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:17:30.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:30.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44115]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:30.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 sudo[44115]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:30.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 ceph-mon[23701]: osdmap e24: 3 total, 3 up, 3 in 2024-09-16T08:17:30.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 ceph-mon[23701]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T08:17:30.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:29 smithi090 ceph-mon[23701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:17:30.249 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 sudo[29397]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:17:30.249 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 sudo[29397]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:30.250 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 sudo[29397]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:30.250 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 sudo[29397]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:30.626 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 sudo[44255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:17:30.627 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 sudo[44255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:30.627 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 sudo[44255]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:30.627 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 sudo[44255]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:17:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:17:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:17:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:17:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:30.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:17:30.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:17:30.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:17:30.880 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 sudo[44259]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:17:30.880 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 sudo[44259]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:17:30.880 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 sudo[44259]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:17:30.880 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 sudo[44259]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:17:30.880 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:17:30.880 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:17:30.881 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:17:30.881 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:17:30.881 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:17:30.881 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:17:30.881 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:17:30.882 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:17:30.882 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:17:30.882 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:17:30.882 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:17:30.882 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:30.883 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:30.883 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:17:30.883 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:30.883 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:17:30.883 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:17:30.884 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:30 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:17:31.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:17:31.126 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:17:31.127 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:31.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:17:31.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:17:31.128 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:30 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:17:32.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[23701]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:17:32.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[23701]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:17:32.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:32.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:32.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:32.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[30018]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:17:32.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[30018]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:17:32.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:32.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:32.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:31 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:32.065 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:31 smithi193 ceph-mon[25614]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:17:32.065 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:31 smithi193 ceph-mon[25614]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:17:32.065 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:31 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:32.065 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:31 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:32.066 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:31 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:33.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2438473920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68684ffb-a84b-4dd1-8b09-b779d0b4b930"}]: dispatch 2024-09-16T08:17:33.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68684ffb-a84b-4dd1-8b09-b779d0b4b930"}]: dispatch 2024-09-16T08:17:33.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68684ffb-a84b-4dd1-8b09-b779d0b4b930"}]': finished 2024-09-16T08:17:33.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[23701]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T08:17:33.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:33.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[23701]: pgmap v89: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:33.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3933925592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:17:33.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2438473920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68684ffb-a84b-4dd1-8b09-b779d0b4b930"}]: dispatch 2024-09-16T08:17:33.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68684ffb-a84b-4dd1-8b09-b779d0b4b930"}]: dispatch 2024-09-16T08:17:33.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68684ffb-a84b-4dd1-8b09-b779d0b4b930"}]': finished 2024-09-16T08:17:33.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[30018]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T08:17:33.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:33.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[30018]: pgmap v89: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:33.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:32 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3933925592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:17:33.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:32 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2438473920' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68684ffb-a84b-4dd1-8b09-b779d0b4b930"}]: dispatch 2024-09-16T08:17:33.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:32 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68684ffb-a84b-4dd1-8b09-b779d0b4b930"}]: dispatch 2024-09-16T08:17:33.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:32 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68684ffb-a84b-4dd1-8b09-b779d0b4b930"}]': finished 2024-09-16T08:17:33.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:32 smithi193 ceph-mon[25614]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T08:17:33.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:32 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:33.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:32 smithi193 ceph-mon[25614]: pgmap v89: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:33.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:32 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3933925592' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:17:35.254 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:35 smithi090 ceph-mon[30018]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 233 KiB/s wr, 16 op/s 2024-09-16T08:17:35.254 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:35 smithi090 ceph-mon[23701]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 233 KiB/s wr, 16 op/s 2024-09-16T08:17:35.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:35 smithi193 ceph-mon[25614]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 233 KiB/s wr, 16 op/s 2024-09-16T08:17:37.297 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:37 smithi090 ceph-mon[30018]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 172 KiB/s wr, 11 op/s 2024-09-16T08:17:37.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:37 smithi090 ceph-mon[23701]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 172 KiB/s wr, 11 op/s 2024-09-16T08:17:37.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:37 smithi193 ceph-mon[25614]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 172 KiB/s wr, 11 op/s 2024-09-16T08:17:38.176 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:38 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T08:17:38.176 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:38 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:38.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T08:17:38.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:38.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:38 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T08:17:38.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:38 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:39.302 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:39 smithi090 ceph-mon[30018]: Deploying daemon osd.3 on smithi090 2024-09-16T08:17:39.302 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:39 smithi090 ceph-mon[30018]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 136 KiB/s wr, 9 op/s 2024-09-16T08:17:39.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:39 smithi090 ceph-mon[23701]: Deploying daemon osd.3 on smithi090 2024-09-16T08:17:39.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:39 smithi090 ceph-mon[23701]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 136 KiB/s wr, 9 op/s 2024-09-16T08:17:39.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:39 smithi193 ceph-mon[25614]: Deploying daemon osd.3 on smithi090 2024-09-16T08:17:39.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:39 smithi193 ceph-mon[25614]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 136 KiB/s wr, 9 op/s 2024-09-16T08:17:41.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:41 smithi090 ceph-mon[30018]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-16T08:17:41.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:41 smithi090 ceph-mon[23701]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-16T08:17:41.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:41 smithi193 ceph-mon[25614]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-16T08:17:43.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:43 smithi090 ceph-mon[30018]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 128 KiB/s wr, 8 op/s 2024-09-16T08:17:43.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:43 smithi090 ceph-mon[23701]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 128 KiB/s wr, 8 op/s 2024-09-16T08:17:43.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:43 smithi193 ceph-mon[25614]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 128 KiB/s wr, 8 op/s 2024-09-16T08:17:44.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:44.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:44.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:44.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:44.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:44.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:44.462 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:44.462 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:44.462 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:44.462 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:44.462 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:44.462 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:44.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:44.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:45.341 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:45 smithi090 ceph-mon[30018]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-16T08:17:45.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:45 smithi090 ceph-mon[23701]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-16T08:17:45.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:45 smithi193 ceph-mon[25614]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-16T08:17:47.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:47.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: from='osd.3 [v2:172.21.15.90:6826/4238944051,v1:172.21.15.90:6827/4238944051]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:17:47.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:17:47.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:47.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:47.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:47.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.180 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:47.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: from='osd.3 [v2:172.21.15.90:6826/4238944051,v1:172.21.15.90:6827/4238944051]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:17:47.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:17:47.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:47.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:47.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:47.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:47 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: from='osd.3 [v2:172.21.15.90:6826/4238944051,v1:172.21.15.90:6827/4238944051]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:17:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:17:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.066 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 3 on host 'smithi090' 2024-09-16T08:17:48.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi090 2024-09-16T08:17:48.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T08:17:48.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: osdmap e26: 4 total, 3 up, 4 in 2024-09-16T08:17:48.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='osd.3 [v2:172.21.15.90:6826/4238944051,v1:172.21.15.90:6827/4238944051]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:48.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:48.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:48.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: osdmap e27: 4 total, 3 up, 4 in 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi090 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: osdmap e26: 4 total, 3 up, 4 in 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='osd.3 [v2:172.21.15.90:6826/4238944051,v1:172.21.15.90:6827/4238944051]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:48.314 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.315 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.316 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.316 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.316 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:48.316 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:48.316 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:17:48.316 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: osdmap e27: 4 total, 3 up, 4 in 2024-09-16T08:17:48.316 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:48 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:48.408 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi090 2024-09-16T08:17:48.408 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T08:17:48.408 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: osdmap e26: 4 total, 3 up, 4 in 2024-09-16T08:17:48.408 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='osd.3 [v2:172.21.15.90:6826/4238944051,v1:172.21.15.90:6827/4238944051]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:48.408 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:17:48.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:48.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:48.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:17:48.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-16T08:17:48.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: osdmap e27: 4 total, 3 up, 4 in 2024-09-16T08:17:48.410 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:48 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:48.695 DEBUG:teuthology.orchestra.run.smithi090:osd.3> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T08:17:48.697 INFO:tasks.cephadm:Deploying osd.4 on smithi193 with /dev/vg_nvme/lv_4... 2024-09-16T08:17:48.698 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T08:17:48.895 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:17:49.321 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:49 smithi193 ceph-mon[25614]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:49.321 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:49 smithi193 ceph-mon[25614]: Adjusting osd_memory_target on smithi090 to 3760M 2024-09-16T08:17:49.321 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:49 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:49.321 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:49 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:49.321 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:49 smithi193 ceph-mon[25614]: from='osd.3 ' entity='osd.3' 2024-09-16T08:17:49.321 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:49 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:49.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[30018]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:49.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[30018]: Adjusting osd_memory_target on smithi090 to 3760M 2024-09-16T08:17:49.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:49.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:49.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[30018]: from='osd.3 ' entity='osd.3' 2024-09-16T08:17:49.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:49.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[23701]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:49.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[23701]: Adjusting osd_memory_target on smithi090 to 3760M 2024-09-16T08:17:49.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:49.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:49.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[23701]: from='osd.3 ' entity='osd.3' 2024-09-16T08:17:49.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:49 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:50.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:50 smithi193 ceph-mon[25614]: purged_snaps scrub starts 2024-09-16T08:17:50.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:50 smithi193 ceph-mon[25614]: purged_snaps scrub ok 2024-09-16T08:17:50.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:50 smithi193 ceph-mon[25614]: osd.3 [v2:172.21.15.90:6826/4238944051,v1:172.21.15.90:6827/4238944051] boot 2024-09-16T08:17:50.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:50 smithi193 ceph-mon[25614]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T08:17:50.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:50 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:50.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[23701]: purged_snaps scrub starts 2024-09-16T08:17:50.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[23701]: purged_snaps scrub ok 2024-09-16T08:17:50.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[23701]: osd.3 [v2:172.21.15.90:6826/4238944051,v1:172.21.15.90:6827/4238944051] boot 2024-09-16T08:17:50.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[23701]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T08:17:50.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:50.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[30018]: purged_snaps scrub starts 2024-09-16T08:17:50.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[30018]: purged_snaps scrub ok 2024-09-16T08:17:50.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[30018]: osd.3 [v2:172.21.15.90:6826/4238944051,v1:172.21.15.90:6827/4238944051] boot 2024-09-16T08:17:50.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[30018]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T08:17:50.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:50 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:17:51.327 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:51 smithi193 ceph-mon[25614]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:51.327 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:51.328 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:51.328 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:51.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:51 smithi090 ceph-mon[23701]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:51.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:51.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:51.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:51.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:51 smithi090 ceph-mon[30018]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:17:51.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:51.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:51.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:17:52.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:52 smithi193 ceph-mon[25614]: osdmap e29: 4 total, 4 up, 4 in 2024-09-16T08:17:52.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:52 smithi090 ceph-mon[23701]: osdmap e29: 4 total, 4 up, 4 in 2024-09-16T08:17:52.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:52 smithi090 ceph-mon[30018]: osdmap e29: 4 total, 4 up, 4 in 2024-09-16T08:17:53.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:53 smithi193 ceph-mon[25614]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:53.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:53 smithi090 ceph-mon[30018]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:53.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:53 smithi090 ceph-mon[23701]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:53.591 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:17:53.610 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-09-16T08:17:53.803 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:17:55.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:55 smithi193 ceph-mon[25614]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:55.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:55 smithi090 ceph-mon[23701]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:55.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:55 smithi090 ceph-mon[30018]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:56.493 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:17:56.493 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:17:56.493 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:56.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:17:56.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:17:56.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:56.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:17:56.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:17:56.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:17:57.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:57 smithi193 ceph-mon[25614]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:17:57.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:57 smithi193 ceph-mon[25614]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:57.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:57 smithi090 ceph-mon[30018]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:17:57.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:57 smithi090 ceph-mon[30018]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:57 smithi090 ceph-mon[23701]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:17:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:57 smithi090 ceph-mon[23701]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:58.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:58 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/655451053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aeb8730-714e-4175-b4c4-f276993b0a16"}]: dispatch 2024-09-16T08:17:58.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:58 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aeb8730-714e-4175-b4c4-f276993b0a16"}]: dispatch 2024-09-16T08:17:58.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:58 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4aeb8730-714e-4175-b4c4-f276993b0a16"}]': finished 2024-09-16T08:17:58.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:58 smithi193 ceph-mon[25614]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T08:17:58.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:58 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:17:58.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:58 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/2178051419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:17:58.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/655451053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aeb8730-714e-4175-b4c4-f276993b0a16"}]: dispatch 2024-09-16T08:17:58.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aeb8730-714e-4175-b4c4-f276993b0a16"}]: dispatch 2024-09-16T08:17:58.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4aeb8730-714e-4175-b4c4-f276993b0a16"}]': finished 2024-09-16T08:17:58.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[23701]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T08:17:58.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:17:58.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/2178051419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:17:58.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/655451053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aeb8730-714e-4175-b4c4-f276993b0a16"}]: dispatch 2024-09-16T08:17:58.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4aeb8730-714e-4175-b4c4-f276993b0a16"}]: dispatch 2024-09-16T08:17:58.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4aeb8730-714e-4175-b4c4-f276993b0a16"}]': finished 2024-09-16T08:17:58.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[30018]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T08:17:58.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:17:58.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:58 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/2178051419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:17:59.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:17:59 smithi193 ceph-mon[25614]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:59.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:17:59 smithi090 ceph-mon[23701]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:17:59.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:17:59 smithi090 ceph-mon[30018]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:01.470 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:01 smithi193 ceph-mon[25614]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:01.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:01 smithi090 ceph-mon[23701]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:01.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:01 smithi090 ceph-mon[30018]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:03.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:03 smithi193 ceph-mon[25614]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:03.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:03.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:03.422 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:03 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:03.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:03 smithi090 ceph-mon[23701]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:03.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:03.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:03.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:03 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:03.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:03 smithi090 ceph-mon[30018]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:03.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:03.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:03.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:03 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:04.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T08:18:04.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T08:18:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:04.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T08:18:04.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:05.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:05 smithi193 ceph-mon[25614]: Deploying daemon osd.4 on smithi193 2024-09-16T08:18:05.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:05 smithi193 ceph-mon[25614]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:05.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:05 smithi090 ceph-mon[23701]: Deploying daemon osd.4 on smithi193 2024-09-16T08:18:05.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:05 smithi090 ceph-mon[23701]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:05.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:05 smithi090 ceph-mon[30018]: Deploying daemon osd.4 on smithi193 2024-09-16T08:18:05.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:05 smithi090 ceph-mon[30018]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:07.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:07 smithi193 ceph-mon[25614]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:07.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:07 smithi090 ceph-mon[23701]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:07.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:07 smithi090 ceph-mon[30018]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:09.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:09 smithi193 ceph-mon[25614]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:09.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:09 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:09.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:09 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:09.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:09 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:09.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:09 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:09.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:09 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:09.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:09 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:09.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[23701]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:09.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:09.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:09.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:09.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:09.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:09.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:09.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[30018]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:09.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:09.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:09.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:09.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:09.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:09.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:09 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:11 smithi193 ceph-mon[25614]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:11.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:11 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[23701]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:11.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:11.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:11.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:11.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[30018]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:11.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:11.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:11.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:11.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:11 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:11.731 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 4 on host 'smithi193' 2024-09-16T08:18:12.386 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:12 smithi193 ceph-mon[25614]: from='osd.4 [v2:172.21.15.193:6800/2641807789,v1:172.21.15.193:6801/2641807789]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:18:12.386 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:12 smithi193 ceph-mon[25614]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:18:12.386 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:12.386 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:12.387 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:18:12.387 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:12.387 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:12.387 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:12.387 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:12 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[23701]: from='osd.4 [v2:172.21.15.193:6800/2641807789,v1:172.21.15.193:6801/2641807789]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:18:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[23701]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:18:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:18:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:12.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[30018]: from='osd.4 [v2:172.21.15.193:6800/2641807789,v1:172.21.15.193:6801/2641807789]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:18:12.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[30018]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:18:12.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:12.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:12.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:18:12.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:12.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:12.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:12.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:12 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:12.760 DEBUG:teuthology.orchestra.run.smithi193:osd.4> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T08:18:12.763 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-09-16T08:18:12.763 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T08:18:13.363 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:18:13.216+0000 7f3e80567640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T08:18:13.363 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:18:13.220+0000 7f3e7c390640 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:18:13.363 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:13.363 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T08:18:13.363 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T08:18:13.363 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:13.363 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='osd.4 [v2:172.21.15.193:6800/2641807789,v1:172.21.15.193:6801/2641807789]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:13.363 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:13.364 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.364 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.364 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:13.364 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.364 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:13.364 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:13.364 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:13 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='osd.4 [v2:172.21.15.193:6800/2641807789,v1:172.21.15.193:6801/2641807789]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='osd.4 [v2:172.21.15.193:6800/2641807789,v1:172.21.15.193:6801/2641807789]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:13.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:13.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:13.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:13 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:13.598 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:18:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:14 smithi193 ceph-mon[25614]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:18:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:14 smithi193 ceph-mon[25614]: osdmap e32: 5 total, 4 up, 5 in 2024-09-16T08:18:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:14 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:14 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:14.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:14 smithi090 ceph-mon[23701]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:18:14.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:14 smithi090 ceph-mon[23701]: osdmap e32: 5 total, 4 up, 5 in 2024-09-16T08:18:14.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:14 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:14.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:14 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:14.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:14 smithi090 ceph-mon[30018]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:18:14.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:14 smithi090 ceph-mon[30018]: osdmap e32: 5 total, 4 up, 5 in 2024-09-16T08:18:14.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:14 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:14.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:14 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:15.344 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:15 smithi193 ceph-mon[25614]: purged_snaps scrub starts 2024-09-16T08:18:15.344 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:15 smithi193 ceph-mon[25614]: purged_snaps scrub ok 2024-09-16T08:18:15.344 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:15 smithi193 ceph-mon[25614]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:15.344 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:15 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:15.344 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:15 smithi193 ceph-mon[25614]: osd.4 [v2:172.21.15.193:6800/2641807789,v1:172.21.15.193:6801/2641807789] boot 2024-09-16T08:18:15.344 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:15 smithi193 ceph-mon[25614]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T08:18:15.344 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:15 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:15.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[23701]: purged_snaps scrub starts 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[23701]: purged_snaps scrub ok 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[23701]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[23701]: osd.4 [v2:172.21.15.193:6800/2641807789,v1:172.21.15.193:6801/2641807789] boot 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[23701]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[30018]: purged_snaps scrub starts 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[30018]: purged_snaps scrub ok 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[30018]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-16T08:18:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:15.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[30018]: osd.4 [v2:172.21.15.193:6800/2641807789,v1:172.21.15.193:6801/2641807789] boot 2024-09-16T08:18:15.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[30018]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T08:18:15.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:15 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:18:16.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:16 smithi193 ceph-mon[25614]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T08:18:16.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:16 smithi090 ceph-mon[30018]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T08:18:16.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:16 smithi090 ceph-mon[23701]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T08:18:16.628 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:18:16.647 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-09-16T08:18:16.839 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:18:17.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:17 smithi193 ceph-mon[25614]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:17.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:17 smithi193 ceph-mon[25614]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T08:18:17.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:17 smithi193 ceph-mon[25614]: osdmap e36: 5 total, 5 up, 5 in 2024-09-16T08:18:17.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:17 smithi090 ceph-mon[30018]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:17.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:17 smithi090 ceph-mon[30018]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T08:18:17.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:17 smithi090 ceph-mon[30018]: osdmap e36: 5 total, 5 up, 5 in 2024-09-16T08:18:17.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:17 smithi090 ceph-mon[23701]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:17.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:17 smithi090 ceph-mon[23701]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T08:18:17.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:17 smithi090 ceph-mon[23701]: osdmap e36: 5 total, 5 up, 5 in 2024-09-16T08:18:19.153 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:19 smithi193 ceph-mon[25614]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:19.154 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:19 smithi193 ceph-mon[25614]: osdmap e37: 5 total, 5 up, 5 in 2024-09-16T08:18:19.154 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:19 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:18:19.154 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:19 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:18:19.154 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:19 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:19.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[23701]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:19.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[23701]: osdmap e37: 5 total, 5 up, 5 in 2024-09-16T08:18:19.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:18:19.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:18:19.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:19.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[30018]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:19.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[30018]: osdmap e37: 5 total, 5 up, 5 in 2024-09-16T08:18:19.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:18:19.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:18:19.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:19 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:20.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:20 smithi193 ceph-mon[25614]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:18:20.311 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:20 smithi090 ceph-mon[30018]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:18:20.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:20 smithi090 ceph-mon[23701]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:18:21.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:21 smithi193 ceph-mon[25614]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-16T08:18:21.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:21 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/3907605693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3192b17d-4647-49ec-9385-a997434b9a64"}]: dispatch 2024-09-16T08:18:21.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:21 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3192b17d-4647-49ec-9385-a997434b9a64"}]: dispatch 2024-09-16T08:18:21.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:21 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3192b17d-4647-49ec-9385-a997434b9a64"}]': finished 2024-09-16T08:18:21.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:21 smithi193 ceph-mon[25614]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T08:18:21.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:21 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:21.305 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[30018]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-16T08:18:21.305 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/3907605693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3192b17d-4647-49ec-9385-a997434b9a64"}]: dispatch 2024-09-16T08:18:21.305 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3192b17d-4647-49ec-9385-a997434b9a64"}]: dispatch 2024-09-16T08:18:21.305 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3192b17d-4647-49ec-9385-a997434b9a64"}]': finished 2024-09-16T08:18:21.305 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[30018]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T08:18:21.305 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:21.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[23701]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-16T08:18:21.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/3907605693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3192b17d-4647-49ec-9385-a997434b9a64"}]: dispatch 2024-09-16T08:18:21.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3192b17d-4647-49ec-9385-a997434b9a64"}]: dispatch 2024-09-16T08:18:21.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3192b17d-4647-49ec-9385-a997434b9a64"}]': finished 2024-09-16T08:18:21.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[23701]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T08:18:21.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:21 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:22.282 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:22 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/2488166101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:18:22.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:22 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/2488166101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:18:22.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:22 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/2488166101' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:18:23.223 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:23 smithi090 ceph-mon[30018]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-16T08:18:23.224 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:23 smithi090 ceph-mon[23701]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-16T08:18:23.391 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:23 smithi193 ceph-mon[25614]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-16T08:18:24.483 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:24.484 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:24.484 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:24.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:24.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:24.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:24.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:24.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:24.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:25.329 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:25 smithi193 ceph-mon[25614]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T08:18:25.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:25 smithi090 ceph-mon[30018]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T08:18:25.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:25 smithi090 ceph-mon[23701]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T08:18:27.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:27 smithi193 ceph-mon[25614]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:18:27.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T08:18:27.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:27 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:27.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:27 smithi090 ceph-mon[30018]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:18:27.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T08:18:27.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:27 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:27.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:27 smithi090 ceph-mon[23701]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:18:27.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T08:18:27.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:27 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:28.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:28 smithi193 ceph-mon[25614]: Deploying daemon osd.5 on smithi193 2024-09-16T08:18:28.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:28 smithi090 ceph-mon[30018]: Deploying daemon osd.5 on smithi193 2024-09-16T08:18:28.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:28 smithi090 ceph-mon[23701]: Deploying daemon osd.5 on smithi193 2024-09-16T08:18:29.457 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:29 smithi193 ceph-mon[25614]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:18:29.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:29 smithi090 ceph-mon[30018]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:18:29.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:29 smithi090 ceph-mon[23701]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:18:31.393 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:31 smithi193 ceph-mon[25614]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:31.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:31 smithi090 ceph-mon[30018]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:31.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:31 smithi090 ceph-mon[23701]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:32.308 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:32 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:32.308 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:32 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:32.308 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:32 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:32.308 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:32 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:32.308 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:32 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:32.308 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:32 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:32.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:32.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:32.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:32.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:32.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:32.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:32.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:32.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:32.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:32.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:32.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:32.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:32 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:33 smithi193 ceph-mon[25614]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:33.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:33 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:33 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:33 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:33.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:33 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:33 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:33.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:33 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:33.409 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:33 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[30018]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[23701]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:33.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:33.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:33.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:33.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:33.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:33 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:34.496 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:34 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi193 2024-09-16T08:18:34.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:34 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi193 2024-09-16T08:18:34.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:34 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi193 2024-09-16T08:18:35.460 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:35 smithi193 ceph-mon[25614]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:35.461 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:35 smithi193 ceph-mon[25614]: from='osd.5 [v2:172.21.15.193:6808/1961012311,v1:172.21.15.193:6809/1961012311]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:18:35.461 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:35 smithi193 ceph-mon[25614]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:18:35.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:35 smithi090 ceph-mon[23701]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:35.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:35 smithi090 ceph-mon[23701]: from='osd.5 [v2:172.21.15.193:6808/1961012311,v1:172.21.15.193:6809/1961012311]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:18:35.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:35 smithi090 ceph-mon[23701]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:18:35.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:35 smithi090 ceph-mon[30018]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:35.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:35 smithi090 ceph-mon[30018]: from='osd.5 [v2:172.21.15.193:6808/1961012311,v1:172.21.15.193:6809/1961012311]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:18:35.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:35 smithi090 ceph-mon[30018]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:18:36.414 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:36 smithi193 ceph-mon[25614]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T08:18:36.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:36 smithi193 ceph-mon[25614]: osdmap e39: 6 total, 5 up, 6 in 2024-09-16T08:18:36.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:36 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:36.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:36 smithi193 ceph-mon[25614]: from='osd.5 [v2:172.21.15.193:6808/1961012311,v1:172.21.15.193:6809/1961012311]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:36.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:36 smithi193 ceph-mon[25614]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:36.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[23701]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T08:18:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[23701]: osdmap e39: 6 total, 5 up, 6 in 2024-09-16T08:18:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[23701]: from='osd.5 [v2:172.21.15.193:6808/1961012311,v1:172.21.15.193:6809/1961012311]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:36.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[23701]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[30018]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T08:18:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[30018]: osdmap e39: 6 total, 5 up, 6 in 2024-09-16T08:18:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[30018]: from='osd.5 [v2:172.21.15.193:6808/1961012311,v1:172.21.15.193:6809/1961012311]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:36 smithi090 ceph-mon[30018]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:18:36.737 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-09-16T08:18:37.311 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:37.311 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: osdmap e40: 6 total, 5 up, 6 in 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.312 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: osd.5 [v2:172.21.15.193:6808/1961012311,v1:172.21.15.193:6809/1961012311] boot 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T08:18:37.313 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:37 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:37.332 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T08:18:37.335 INFO:tasks.cephadm:Deploying osd.6 on smithi193 with /dev/vg_nvme/lv_2... 2024-09-16T08:18:37.335 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T08:18:37.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:37.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:18:37.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: osdmap e40: 6 total, 5 up, 6 in 2024-09-16T08:18:37.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:37.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:37.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: osd.5 [v2:172.21.15.193:6808/1961012311,v1:172.21.15.193:6809/1961012311] boot 2024-09-16T08:18:37.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T08:18:37.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:37.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:18:37.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:18:37.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: osdmap e40: 6 total, 5 up, 6 in 2024-09-16T08:18:37.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:37.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:37.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:18:37.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:37.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:37.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:37.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: osd.5 [v2:172.21.15.193:6808/1961012311,v1:172.21.15.193:6809/1961012311] boot 2024-09-16T08:18:37.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T08:18:37.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:37 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:18:37.566 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:18:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[23701]: purged_snaps scrub starts 2024-09-16T08:18:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[23701]: purged_snaps scrub ok 2024-09-16T08:18:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi193 2024-09-16T08:18:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[23701]: Adjusting osd_memory_target on smithi193 to 8032M 2024-09-16T08:18:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[23701]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T08:18:38.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[30018]: purged_snaps scrub starts 2024-09-16T08:18:38.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[30018]: purged_snaps scrub ok 2024-09-16T08:18:38.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi193 2024-09-16T08:18:38.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[30018]: Adjusting osd_memory_target on smithi193 to 8032M 2024-09-16T08:18:38.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:38 smithi090 ceph-mon[30018]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T08:18:38.642 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:38 smithi193 ceph-mon[25614]: purged_snaps scrub starts 2024-09-16T08:18:38.642 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:38 smithi193 ceph-mon[25614]: purged_snaps scrub ok 2024-09-16T08:18:38.642 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:38 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi193 2024-09-16T08:18:38.642 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:38 smithi193 ceph-mon[25614]: Adjusting osd_memory_target on smithi193 to 8032M 2024-09-16T08:18:38.642 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:38 smithi193 ceph-mon[25614]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T08:18:39.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:39 smithi090 ceph-mon[23701]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:39 smithi090 ceph-mon[23701]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T08:18:39.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:39 smithi090 ceph-mon[30018]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:39.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:39 smithi090 ceph-mon[30018]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T08:18:39.747 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:39 smithi193 ceph-mon[25614]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:39.748 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:39 smithi193 ceph-mon[25614]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T08:18:40.036 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:18:40.055 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-09-16T08:18:40.250 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:18:41.272 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:41 smithi193 ceph-mon[25614]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:41.272 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:41 smithi193 ceph-mon[25614]: osdmap e44: 6 total, 6 up, 6 in 2024-09-16T08:18:41.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:41 smithi090 ceph-mon[23701]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:41.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:41 smithi090 ceph-mon[23701]: osdmap e44: 6 total, 6 up, 6 in 2024-09-16T08:18:41.299 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:41 smithi090 ceph-mon[30018]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:41.299 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:41 smithi090 ceph-mon[30018]: osdmap e44: 6 total, 6 up, 6 in 2024-09-16T08:18:42.143 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:42 smithi090 ceph-mon[30018]: osdmap e45: 6 total, 6 up, 6 in 2024-09-16T08:18:42.143 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:42 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:18:42.143 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:42 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:18:42.143 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:42 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:42.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:42 smithi090 ceph-mon[23701]: osdmap e45: 6 total, 6 up, 6 in 2024-09-16T08:18:42.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:42 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:18:42.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:42 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:18:42.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:42 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:42.382 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:42 smithi193 ceph-mon[25614]: osdmap e45: 6 total, 6 up, 6 in 2024-09-16T08:18:42.382 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:18:42.382 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:18:42.382 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:42 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:43.198 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:43 smithi193 ceph-mon[25614]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:18:43.198 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:43 smithi193 ceph-mon[25614]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:43.311 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:43 smithi090 ceph-mon[30018]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:18:43.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:43 smithi090 ceph-mon[30018]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:43.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:43 smithi090 ceph-mon[23701]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:18:43.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:43 smithi090 ceph-mon[23701]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:44.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:44 smithi193 ceph-mon[25614]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:44 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/3971087674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3357150d-8a24-4f20-9778-d5d345a7ed94"}]: dispatch 2024-09-16T08:18:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:44 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3357150d-8a24-4f20-9778-d5d345a7ed94"}]: dispatch 2024-09-16T08:18:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:44 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3357150d-8a24-4f20-9778-d5d345a7ed94"}]': finished 2024-09-16T08:18:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:44 smithi193 ceph-mon[25614]: osdmap e46: 7 total, 6 up, 7 in 2024-09-16T08:18:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:18:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[30018]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/3971087674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3357150d-8a24-4f20-9778-d5d345a7ed94"}]: dispatch 2024-09-16T08:18:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3357150d-8a24-4f20-9778-d5d345a7ed94"}]: dispatch 2024-09-16T08:18:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3357150d-8a24-4f20-9778-d5d345a7ed94"}]': finished 2024-09-16T08:18:44.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[30018]: osdmap e46: 7 total, 6 up, 7 in 2024-09-16T08:18:44.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:18:44.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:44.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:44.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:44.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[23701]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:44.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/3971087674' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3357150d-8a24-4f20-9778-d5d345a7ed94"}]: dispatch 2024-09-16T08:18:44.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3357150d-8a24-4f20-9778-d5d345a7ed94"}]: dispatch 2024-09-16T08:18:44.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3357150d-8a24-4f20-9778-d5d345a7ed94"}]': finished 2024-09-16T08:18:44.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[23701]: osdmap e46: 7 total, 6 up, 7 in 2024-09-16T08:18:44.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:18:45.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:45 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/1680630661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:18:45.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:45 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/1680630661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:18:45.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:45 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/1680630661' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:18:46.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:46 smithi193 ceph-mon[25614]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:46.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:46 smithi090 ceph-mon[23701]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:46.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:46 smithi090 ceph-mon[30018]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:49.244 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:49 smithi193 ceph-mon[25614]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:49.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:49 smithi090 ceph-mon[23701]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:49.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:49 smithi090 ceph-mon[30018]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:51.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:51 smithi193 ceph-mon[25614]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:51.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T08:18:51.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:51.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:51 smithi193 ceph-mon[25614]: Deploying daemon osd.6 on smithi193 2024-09-16T08:18:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:51 smithi090 ceph-mon[23701]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T08:18:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:51.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:51 smithi090 ceph-mon[23701]: Deploying daemon osd.6 on smithi193 2024-09-16T08:18:51.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:51 smithi090 ceph-mon[30018]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:51.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T08:18:51.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:51.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:51 smithi090 ceph-mon[30018]: Deploying daemon osd.6 on smithi193 2024-09-16T08:18:53.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:53 smithi090 ceph-mon[23701]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:53.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:53 smithi090 ceph-mon[30018]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:53.438 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:53 smithi193 ceph-mon[25614]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:55.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:55 smithi090 ceph-mon[30018]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:55.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:55 smithi090 ceph-mon[23701]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:55.347 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:55 smithi193 ceph-mon[25614]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:56.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:56.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:56.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:56.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:56.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:56.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:56 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:56.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:56.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:56.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:56.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:56.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:56.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:56.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:56.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:56.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:56.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:56.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:56.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:56 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:57 smithi193 ceph-mon[25614]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:57.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:57.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:57.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:57.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:57 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[23701]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[30018]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:57 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:59.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi193 2024-09-16T08:18:59.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:59.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.402 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:59.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:59.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: from='osd.6 [v2:172.21.15.193:6816/623377406,v1:172.21.15.193:6817/623377406]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T08:18:59.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:18:59 smithi193 ceph-mon[25614]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi193 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: from='osd.6 [v2:172.21.15.193:6816/623377406,v1:172.21.15.193:6817/623377406]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T08:18:59.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[23701]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi193 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:18:59.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: from='osd.6 [v2:172.21.15.193:6816/623377406,v1:172.21.15.193:6817/623377406]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T08:18:59.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:18:59 smithi090 ceph-mon[30018]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T08:18:59.946 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 6 on host 'smithi193' 2024-09-16T08:19:00.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: osdmap e47: 7 total, 6 up, 7 in 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='osd.6 [v2:172.21.15.193:6816/623377406,v1:172.21.15.193:6817/623377406]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:00.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:00.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:00 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T08:19:00.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: osdmap e47: 7 total, 6 up, 7 in 2024-09-16T08:19:00.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:00.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='osd.6 [v2:172.21.15.193:6816/623377406,v1:172.21.15.193:6817/623377406]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:00.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:00.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:00.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T08:19:00.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: osdmap e47: 7 total, 6 up, 7 in 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='osd.6 [v2:172.21.15.193:6816/623377406,v1:172.21.15.193:6817/623377406]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:00.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:00.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:00 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:00.604 DEBUG:teuthology.orchestra.run.smithi193:osd.6> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T08:19:00.606 INFO:tasks.cephadm:Deploying osd.7 on smithi193 with /dev/vg_nvme/lv_1... 2024-09-16T08:19:00.607 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T08:19:00.840 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:19:01.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:01 smithi193 ceph-mon[25614]: Adjusting osd_memory_target on smithi193 to 5354M 2024-09-16T08:19:01.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:01 smithi193 ceph-mon[25614]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:19:01.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:01 smithi193 ceph-mon[25614]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:19:01.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:01 smithi193 ceph-mon[25614]: osdmap e48: 7 total, 6 up, 7 in 2024-09-16T08:19:01.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:01 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:01.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:01 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:01.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:01 smithi193 ceph-mon[25614]: from='osd.6 ' entity='osd.6' 2024-09-16T08:19:01.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[23701]: Adjusting osd_memory_target on smithi193 to 5354M 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[23701]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[23701]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[23701]: osdmap e48: 7 total, 6 up, 7 in 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[23701]: from='osd.6 ' entity='osd.6' 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[30018]: Adjusting osd_memory_target on smithi193 to 5354M 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[30018]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[30018]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[30018]: osdmap e48: 7 total, 6 up, 7 in 2024-09-16T08:19:01.351 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:01.352 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:01.352 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:01 smithi090 ceph-mon[30018]: from='osd.6 ' entity='osd.6' 2024-09-16T08:19:02.288 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[30018]: purged_snaps scrub starts 2024-09-16T08:19:02.288 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[30018]: purged_snaps scrub ok 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[30018]: osd.6 [v2:172.21.15.193:6816/623377406,v1:172.21.15.193:6817/623377406] boot 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[30018]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[30018]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[23701]: purged_snaps scrub starts 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[23701]: purged_snaps scrub ok 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[23701]: osd.6 [v2:172.21.15.193:6816/623377406,v1:172.21.15.193:6817/623377406] boot 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[23701]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:02.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:02 smithi090 ceph-mon[23701]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T08:19:02.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:02 smithi193 ceph-mon[25614]: purged_snaps scrub starts 2024-09-16T08:19:02.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:02 smithi193 ceph-mon[25614]: purged_snaps scrub ok 2024-09-16T08:19:02.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:02 smithi193 ceph-mon[25614]: osd.6 [v2:172.21.15.193:6816/623377406,v1:172.21.15.193:6817/623377406] boot 2024-09-16T08:19:02.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:02 smithi193 ceph-mon[25614]: osdmap e49: 7 total, 7 up, 7 in 2024-09-16T08:19:02.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:02 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:02.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:02 smithi193 ceph-mon[25614]: osdmap e50: 7 total, 7 up, 7 in 2024-09-16T08:19:03.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:03 smithi193 ceph-mon[25614]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:03.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:03 smithi193 ceph-mon[25614]: osdmap e51: 7 total, 7 up, 7 in 2024-09-16T08:19:03.362 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-09-16T08:19:03.381 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-09-16T08:19:03.460 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:03 smithi090 ceph-mon[30018]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:03.460 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:03 smithi090 ceph-mon[30018]: osdmap e51: 7 total, 7 up, 7 in 2024-09-16T08:19:03.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:03 smithi090 ceph-mon[23701]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:03.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:03 smithi090 ceph-mon[23701]: osdmap e51: 7 total, 7 up, 7 in 2024-09-16T08:19:03.578 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:19:04.737 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:04.738 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:04.738 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:04 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:04.738 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:04 smithi193 ceph-mon[25614]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:04.738 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:04 smithi193 ceph-mon[25614]: osdmap e52: 7 total, 7 up, 7 in 2024-09-16T08:19:04.811 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:04.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:04.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:04.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[30018]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:04.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[30018]: osdmap e52: 7 total, 7 up, 7 in 2024-09-16T08:19:04.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:04.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:04.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:04.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[23701]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:04.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:04 smithi090 ceph-mon[23701]: osdmap e52: 7 total, 7 up, 7 in 2024-09-16T08:19:06.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:06 smithi193 ceph-mon[25614]: osdmap e53: 7 total, 7 up, 7 in 2024-09-16T08:19:06.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:06 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:19:06.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:06 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:19:06.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:06 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:06.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:06 smithi090 ceph-mon[23701]: osdmap e53: 7 total, 7 up, 7 in 2024-09-16T08:19:06.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:06 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:19:06.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:06 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:19:06.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:06 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:06.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:06 smithi090 ceph-mon[30018]: osdmap e53: 7 total, 7 up, 7 in 2024-09-16T08:19:06.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:06 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:19:06.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:06 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:19:06.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:06 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:07.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:07 smithi193 ceph-mon[25614]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:07.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:07 smithi193 ceph-mon[25614]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:07.311 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:07 smithi090 ceph-mon[30018]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:07.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:07 smithi090 ceph-mon[30018]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:07.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:07 smithi090 ceph-mon[23701]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:07.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:07 smithi090 ceph-mon[23701]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:08.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:08 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/3505218135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bb59f2b-d01f-4f5e-80d6-a4bec26999db"}]: dispatch 2024-09-16T08:19:08.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:08 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bb59f2b-d01f-4f5e-80d6-a4bec26999db"}]: dispatch 2024-09-16T08:19:08.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:08 smithi193 ceph-mon[25614]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bb59f2b-d01f-4f5e-80d6-a4bec26999db"}]': finished 2024-09-16T08:19:08.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:08 smithi193 ceph-mon[25614]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T08:19:08.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:08 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:08.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/3505218135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bb59f2b-d01f-4f5e-80d6-a4bec26999db"}]: dispatch 2024-09-16T08:19:08.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bb59f2b-d01f-4f5e-80d6-a4bec26999db"}]: dispatch 2024-09-16T08:19:08.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[23701]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bb59f2b-d01f-4f5e-80d6-a4bec26999db"}]': finished 2024-09-16T08:19:08.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[23701]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T08:19:08.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:08.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/3505218135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bb59f2b-d01f-4f5e-80d6-a4bec26999db"}]: dispatch 2024-09-16T08:19:08.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bb59f2b-d01f-4f5e-80d6-a4bec26999db"}]: dispatch 2024-09-16T08:19:08.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[30018]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bb59f2b-d01f-4f5e-80d6-a4bec26999db"}]': finished 2024-09-16T08:19:08.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[30018]: osdmap e54: 8 total, 7 up, 8 in 2024-09-16T08:19:08.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:08 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:09.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:09 smithi193 ceph-mon[25614]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-16T08:19:09.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:09 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/30026395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:19:09.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:09 smithi090 ceph-mon[30018]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-16T08:19:09.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:09 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/30026395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:19:09.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:09 smithi090 ceph-mon[23701]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-16T08:19:09.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:09 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/30026395' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:19:11.452 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:11 smithi193 ceph-mon[25614]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T08:19:11.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:11 smithi090 ceph-mon[23701]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T08:19:11.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:11 smithi090 ceph-mon[30018]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T08:19:13.436 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:13 smithi193 ceph-mon[25614]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T08:19:13.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:13 smithi090 ceph-mon[23701]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T08:19:13.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:13 smithi090 ceph-mon[30018]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T08:19:14.454 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:14 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T08:19:14.455 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:14 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:14.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:14 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T08:19:14.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:14 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:14.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:14 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T08:19:14.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:14 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:15.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:15 smithi193 ceph-mon[25614]: Deploying daemon osd.7 on smithi193 2024-09-16T08:19:15.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:15 smithi193 ceph-mon[25614]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-16T08:19:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:15 smithi090 ceph-mon[30018]: Deploying daemon osd.7 on smithi193 2024-09-16T08:19:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:15 smithi090 ceph-mon[30018]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-16T08:19:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:15 smithi090 ceph-mon[23701]: Deploying daemon osd.7 on smithi193 2024-09-16T08:19:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:15 smithi090 ceph-mon[23701]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-16T08:19:16.489 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:16 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:16.490 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:16 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:16.490 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:16 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:16.490 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:16 smithi193 ceph-mon[25614]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:19:16.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:16 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:16.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:16 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:16.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:16 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:16.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:16 smithi090 ceph-mon[30018]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:19:16.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:16 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:16.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:16 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:16.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:16 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:16.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:16 smithi090 ceph-mon[23701]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:19:19.131 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:19 smithi193 ceph-mon[25614]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:19.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:19 smithi090 ceph-mon[23701]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:19.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:19 smithi090 ceph-mon[30018]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:20.195 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:20.196 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:20.196 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:20.196 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:20.196 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:20.196 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:20 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:20.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:20.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:20.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:20.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:20.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:20.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:20.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:20 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:21.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:21 smithi193 ceph-mon[25614]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:21.509 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:21 smithi090 ceph-mon[30018]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:21.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:21 smithi090 ceph-mon[23701]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:23.081 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:23 smithi090 ceph-mon[30018]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:23.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:23 smithi090 ceph-mon[23701]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:23.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:23 smithi090 ceph-mon[23701]: from='osd.7 [v2:172.21.15.193:6824/424442086,v1:172.21.15.193:6825/424442086]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T08:19:23.187 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:23 smithi193 ceph-mon[25614]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:23.187 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:23 smithi193 ceph-mon[25614]: from='osd.7 [v2:172.21.15.193:6824/424442086,v1:172.21.15.193:6825/424442086]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T08:19:23.187 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:23 smithi193 ceph-mon[25614]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T08:19:23.221 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 7 on host 'smithi193' 2024-09-16T08:19:23.496 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:23 smithi090 ceph-mon[30018]: from='osd.7 [v2:172.21.15.193:6824/424442086,v1:172.21.15.193:6825/424442086]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T08:19:23.496 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:23 smithi090 ceph-mon[30018]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T08:19:23.496 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:23 smithi090 ceph-mon[23701]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T08:19:23.828 DEBUG:teuthology.orchestra.run.smithi193:osd.7> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T08:19:23.831 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-16T08:19:23.831 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T08:19:24.033 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:19:24.155 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T08:19:24.155 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: osdmap e55: 8 total, 7 up, 8 in 2024-09-16T08:19:24.155 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:24.155 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='osd.7 [v2:172.21.15.193:6824/424442086,v1:172.21.15.193:6825/424442086]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:24.155 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:24.155 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: Adjusting osd_memory_target on smithi193 to 4016M 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.156 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: osdmap e55: 8 total, 7 up, 8 in 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='osd.7 [v2:172.21.15.193:6824/424442086,v1:172.21.15.193:6825/424442086]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: Adjusting osd_memory_target on smithi193 to 4016M 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:24 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T08:19:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: osdmap e55: 8 total, 7 up, 8 in 2024-09-16T08:19:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='osd.7 [v2:172.21.15.193:6824/424442086,v1:172.21.15.193:6825/424442086]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-09-16T08:19:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: Adjusting osd_memory_target on smithi193 to 4016M 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:25.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:19:24.523+0000 7f119090f640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T08:19:25.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:19:24 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:19:24.532+0000 7f1188730640 -1 osd.7 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:19:25.414 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[23701]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:25.414 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[23701]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:19:25.414 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[23701]: osdmap e56: 8 total, 7 up, 8 in 2024-09-16T08:19:25.414 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:25.414 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:25.414 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[23701]: from='osd.7 ' entity='osd.7' 2024-09-16T08:19:25.414 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[23701]: osd.7 [v2:172.21.15.193:6824/424442086,v1:172.21.15.193:6825/424442086] boot 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[23701]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[30018]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[30018]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[30018]: osdmap e56: 8 total, 7 up, 8 in 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[30018]: from='osd.7 ' entity='osd.7' 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[30018]: osd.7 [v2:172.21.15.193:6824/424442086,v1:172.21.15.193:6825/424442086] boot 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[30018]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T08:19:25.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:25 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:25 smithi193 ceph-mon[25614]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-09-16T08:19:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:25 smithi193 ceph-mon[25614]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-09-16T08:19:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:25 smithi193 ceph-mon[25614]: osdmap e56: 8 total, 7 up, 8 in 2024-09-16T08:19:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:25 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:25 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:25 smithi193 ceph-mon[25614]: from='osd.7 ' entity='osd.7' 2024-09-16T08:19:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:25 smithi193 ceph-mon[25614]: osd.7 [v2:172.21.15.193:6824/424442086,v1:172.21.15.193:6825/424442086] boot 2024-09-16T08:19:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:25 smithi193 ceph-mon[25614]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T08:19:25.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:25 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:25.766 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:19:26.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:26 smithi090 ceph-mon[23701]: purged_snaps scrub starts 2024-09-16T08:19:26.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:26 smithi090 ceph-mon[23701]: purged_snaps scrub ok 2024-09-16T08:19:26.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:26 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/474757430' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T08:19:26.366 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:26 smithi090 ceph-mon[30018]: purged_snaps scrub starts 2024-09-16T08:19:26.366 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:26 smithi090 ceph-mon[30018]: purged_snaps scrub ok 2024-09-16T08:19:26.366 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:26 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/474757430' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T08:19:26.387 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":57,"num_osds":8,"num_up_osds":8,"osd_up_since":1726474765,"num_in_osds":8,"osd_in_since":1726474747,"num_remapped_pgs":1} 2024-09-16T08:19:26.387 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T08:19:26.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:26 smithi193 ceph-mon[25614]: purged_snaps scrub starts 2024-09-16T08:19:26.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:26 smithi193 ceph-mon[25614]: purged_snaps scrub ok 2024-09-16T08:19:26.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:26 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/474757430' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T08:19:26.582 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:19:27.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:27 smithi090 ceph-mon[30018]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:27.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:27 smithi090 ceph-mon[30018]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T08:19:27.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:27 smithi090 ceph-mon[30018]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T08:19:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:27 smithi090 ceph-mon[23701]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:27 smithi090 ceph-mon[23701]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T08:19:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:27 smithi090 ceph-mon[23701]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T08:19:27.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:27 smithi193 ceph-mon[25614]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:27.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:27 smithi193 ceph-mon[25614]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T08:19:27.403 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:27 smithi193 ceph-mon[25614]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T08:19:28.215 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:19:28.215 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":60,"fsid":"8b378aa4-7403-11ef-bceb-c7b262605968","created":"2024-09-16T08:14:17.250796+0000","modified":"2024-09-16T08:19:28.089394+0000","last_up_change":"2024-09-16T08:19:25.082991+0000","last_in_change":"2024-09-16T08:19:07.667092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":19,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T08:17:26.343055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d5e8461-ab0d-447a-9c03-fbf862340931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":10,"up_from":13,"up_thru":59,"down_at":11,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6803","nonce":2713629494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6805","nonce":2713629494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6809","nonce":2713629494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6807","nonce":2713629494}]},"public_addr":"172.21.15.90:6803/2713629494","cluster_addr":"172.21.15.90:6805/2713629494","heartbeat_back_addr":"172.21.15.90:6809/2713629494","heartbeat_front_addr":"172.21.15.90:6807/2713629494","state":["exists","up"]},{"osd":1,"uuid":"18d07a14-8eb1-484a-a16b-6be128901ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6811","nonce":3883828403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6813","nonce":3883828403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6817","nonce":3883828403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6815","nonce":3883828403}]},"public_addr":"172.21.15.90:6811/3883828403","cluster_addr":"172.21.15.90:6813/3883828403","heartbeat_back_addr":"172.21.15.90:6817/3883828403","heartbeat_front_addr":"172.21.15.90:6815/3883828403","state":["exists","up"]},{"osd":2,"uuid":"55d2c1c2-391b-404c-aa8e-4955fc69038f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6819","nonce":1303698361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6821","nonce":1303698361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6825","nonce":1303698361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6823","nonce":1303698361}]},"public_addr":"172.21.15.90:6819/1303698361","cluster_addr":"172.21.15.90:6821/1303698361","heartbeat_back_addr":"172.21.15.90:6825/1303698361","heartbeat_front_addr":"172.21.15.90:6823/1303698361","state":["exists","up"]},{"osd":3,"uuid":"68684ffb-a84b-4dd1-8b09-b779d0b4b930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6827","nonce":4238944051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6829","nonce":4238944051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6833","nonce":4238944051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6831","nonce":4238944051}]},"public_addr":"172.21.15.90:6827/4238944051","cluster_addr":"172.21.15.90:6829/4238944051","heartbeat_back_addr":"172.21.15.90:6833/4238944051","heartbeat_front_addr":"172.21.15.90:6831/4238944051","state":["exists","up"]},{"osd":4,"uuid":"4aeb8730-714e-4175-b4c4-f276993b0a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6801","nonce":2641807789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6803","nonce":2641807789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6807","nonce":2641807789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6805","nonce":2641807789}]},"public_addr":"172.21.15.193:6801/2641807789","cluster_addr":"172.21.15.193:6803/2641807789","heartbeat_back_addr":"172.21.15.193:6807/2641807789","heartbeat_front_addr":"172.21.15.193:6805/2641807789","state":["exists","up"]},{"osd":5,"uuid":"3192b17d-4647-49ec-9385-a997434b9a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6809","nonce":1961012311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6811","nonce":1961012311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6815","nonce":1961012311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6813","nonce":1961012311}]},"public_addr":"172.21.15.193:6809/1961012311","cluster_addr":"172.21.15.193:6811/1961012311","heartbeat_back_addr":"172.21.15.193:6815/1961012311","heartbeat_front_addr":"172.21.15.193:6813/1961012311","state":["exists","up"]},{"osd":6,"uuid":"3357150d-8a24-4f20-9778-d5d345a7ed94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6817","nonce":623377406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6819","nonce":623377406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6823","nonce":623377406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6821","nonce":623377406}]},"public_addr":"172.21.15.193:6817/623377406","cluster_addr":"172.21.15.193:6819/623377406","heartbeat_back_addr":"172.21.15.193:6823/623377406","heartbeat_front_addr":"172.21.15.193:6821/623377406","state":["exists","up"]},{"osd":7,"uuid":"2bb59f2b-d01f-4f5e-80d6-a4bec26999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6825","nonce":424442086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6827","nonce":424442086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6831","nonce":424442086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6829","nonce":424442086}]},"public_addr":"172.21.15.193:6825/424442086","cluster_addr":"172.21.15.193:6827/424442086","heartbeat_back_addr":"172.21.15.193:6831/424442086","heartbeat_front_addr":"172.21.15.193:6829/424442086","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:16:30.515585+0000","dead_epoch":10},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:17:00.317818+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:17:23.471718+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:17:47.493781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:18:12.442823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:18:35.677918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:18:59.853193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:19:23.825797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6801/2155974574":"2024-09-17T08:15:10.045545+0000","172.21.15.90:6800/2155974574":"2024-09-17T08:15:10.045545+0000","172.21.15.90:0/3754108410":"2024-09-17T08:15:10.045545+0000","172.21.15.90:0/3342887308":"2024-09-17T08:15:10.045545+0000","172.21.15.90:0/392352616":"2024-09-17T08:15:10.045545+0000","172.21.15.90:6801/688775638":"2024-09-17T08:14:38.353619+0000","172.21.15.90:6800/688775638":"2024-09-17T08:14:38.353619+0000","172.21.15.90:0/2766943845":"2024-09-17T08:14:38.353619+0000","172.21.15.90:0/2556824909":"2024-09-17T08:14:38.353619+0000","172.21.15.90:0/2522025803":"2024-09-17T08:14:38.353619+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T08:19:28.739 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T08:17:26.343055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T08:19:28.739 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T08:19:28.931 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:19:29.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:29 smithi193 ceph-mon[25614]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:29.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:29 smithi193 ceph-mon[25614]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T08:19:29.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:29 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/4193073475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:19:29.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:29 smithi090 ceph-mon[23701]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:29.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:29 smithi090 ceph-mon[23701]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T08:19:29.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:29 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/4193073475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:19:29.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:29 smithi090 ceph-mon[30018]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:29.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:29 smithi090 ceph-mon[30018]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T08:19:29.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:29 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/4193073475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:19:30.276 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:30 smithi090 ceph-mon[30018]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T08:19:30.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:30 smithi090 ceph-mon[23701]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T08:19:30.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:30 smithi193 ceph-mon[25614]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T08:19:30.649 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-09-16T08:19:31.304 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:31 smithi090 ceph-mon[30018]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:31.304 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:31 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3303433378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T08:19:31.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:31 smithi090 ceph-mon[23701]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:31.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:31 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3303433378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T08:19:31.329 INFO:tasks.cephadm:Adding prometheus.a on smithi193 2024-09-16T08:19:31.330 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi193=a' 2024-09-16T08:19:31.358 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:31 smithi193 ceph-mon[25614]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:31.358 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:31 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3303433378' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T08:19:31.538 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:19:33.200 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled prometheus update... 2024-09-16T08:19:33.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:33 smithi193 ceph-mon[25614]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T08:19:33.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:33 smithi090 ceph-mon[23701]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T08:19:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:33 smithi090 ceph-mon[30018]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-16T08:19:33.809 DEBUG:teuthology.orchestra.run.smithi193:prometheus.a> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-16T08:19:33.852 INFO:tasks.cephadm:Adding node-exporter.a on smithi090 2024-09-16T08:19:33.852 INFO:tasks.cephadm:Adding node-exporter.b on smithi193 2024-09-16T08:19:33.852 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi090=a;smithi193=b' 2024-09-16T08:19:34.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[30018]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:34.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[30018]: Saving service prometheus spec with placement smithi193=a;count:1 2024-09-16T08:19:34.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:34.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:34.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:34.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:34.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:34.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[23701]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:34.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[23701]: Saving service prometheus spec with placement smithi193=a;count:1 2024-09-16T08:19:34.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:34.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:34.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:34.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:34.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:34 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:34.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:34 smithi193 ceph-mon[25614]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:34.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:34 smithi193 ceph-mon[25614]: Saving service prometheus spec with placement smithi193=a;count:1 2024-09-16T08:19:34.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:34 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:34.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:34 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:34.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:34 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:34.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:34 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:34.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:34 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:34.829 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:19:35.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:35 smithi193 ceph-mon[25614]: Deploying daemon prometheus.a on smithi193 2024-09-16T08:19:35.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:35 smithi193 ceph-mon[25614]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T08:19:35.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:35 smithi090 ceph-mon[30018]: Deploying daemon prometheus.a on smithi193 2024-09-16T08:19:35.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:35 smithi090 ceph-mon[30018]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T08:19:35.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:35 smithi090 ceph-mon[23701]: Deploying daemon prometheus.a on smithi193 2024-09-16T08:19:35.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:35 smithi090 ceph-mon[23701]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-16T08:19:36.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:36 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:36.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:36 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:36.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:36 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi193 2024-09-16T08:19:36.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:36 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:36.893 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:36 smithi193 ceph-mon[25614]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:19:37.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:37.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:37.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi193 2024-09-16T08:19:37.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:37.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[30018]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:19:37.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:37.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:37.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi193 2024-09-16T08:19:37.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:37.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:36 smithi090 ceph-mon[23701]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:19:38.365 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled node-exporter update... 2024-09-16T08:19:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[23701]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:19:38.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[30018]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:19:38.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:38 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:38 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:38 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:38 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:38 smithi193 ceph-mon[25614]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:19:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:38 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:39 smithi193 ceph-mon[25614]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi090=a;smithi193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:39 smithi193 ceph-mon[25614]: Saving service node-exporter spec with placement smithi090=a;smithi193=b;count:2 2024-09-16T08:19:39.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:39 smithi090 ceph-mon[23701]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi090=a;smithi193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:39.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:39 smithi090 ceph-mon[23701]: Saving service node-exporter spec with placement smithi090=a;smithi193=b;count:2 2024-09-16T08:19:39.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:39 smithi090 ceph-mon[30018]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi090=a;smithi193=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:39.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:39 smithi090 ceph-mon[30018]: Saving service node-exporter spec with placement smithi090=a;smithi193=b;count:2 2024-09-16T08:19:40.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:40 smithi193 ceph-mon[25614]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-16T08:19:40.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:40 smithi090 ceph-mon[23701]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-16T08:19:40.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:40 smithi090 ceph-mon[30018]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-16T08:19:40.964 DEBUG:teuthology.orchestra.run.smithi090:node-exporter.a> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-16T08:19:40.968 DEBUG:teuthology.orchestra.run.smithi193:node-exporter.b> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-16T08:19:40.971 INFO:tasks.cephadm:Adding alertmanager.a on smithi090 2024-09-16T08:19:40.971 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi090=a' 2024-09-16T08:19:41.259 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:19:43.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:43 smithi193 ceph-mon[25614]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T08:19:43.301 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:43 smithi090 ceph-mon[30018]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T08:19:43.301 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:43 smithi090 ceph-mon[23701]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-16T08:19:44.811 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:44 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:44.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:44 smithi090 ceph-mon[30018]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:44.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:44.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:44.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:44 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:44.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:44 smithi090 ceph-mon[23701]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:45.001 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:45.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:45.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:44 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:45.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:44 smithi193 ceph-mon[25614]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:47.021 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled alertmanager update... 2024-09-16T08:19:47.447 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:47 smithi193 ceph-mon[25614]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:47.447 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:47 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:47.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:47 smithi090 ceph-mon[23701]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:47.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:47 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:47.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:47 smithi090 ceph-mon[30018]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:47.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:47 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:47.705 DEBUG:teuthology.orchestra.run.smithi090:alertmanager.a> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-16T08:19:47.709 INFO:tasks.cephadm:Adding grafana.a on smithi193 2024-09-16T08:19:47.709 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi193=a' 2024-09-16T08:19:48.236 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:19:48.355 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:48 smithi193 ceph-mon[25614]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:48.355 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:48 smithi193 ceph-mon[25614]: Saving service alertmanager spec with placement smithi090=a;count:1 2024-09-16T08:19:48.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:48 smithi090 ceph-mon[23701]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:48.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:48 smithi090 ceph-mon[23701]: Saving service alertmanager spec with placement smithi090=a;count:1 2024-09-16T08:19:48.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:48 smithi090 ceph-mon[30018]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi090=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:48.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:48 smithi090 ceph-mon[30018]: Saving service alertmanager spec with placement smithi090=a;count:1 2024-09-16T08:19:49.394 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:49 smithi193 ceph-mon[25614]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:49.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:49 smithi090 ceph-mon[30018]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:49.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:49 smithi090 ceph-mon[23701]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:49.991 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:49 smithi193 podman[44822]: 2024-09-16 08:19:49.746107009 +0000 UTC m=+0.018452528 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-16T08:19:49.992 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:49 smithi193 podman[44822]: 2024-09-16 08:19:49.991862863 +0000 UTC m=+0.264208366 container create 9b7edc7a6ae843e78ac9085d231f2cfb1b39e9b029a6a1a6dd10ab370bfdac1a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:19:50.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 podman[44822]: 2024-09-16 08:19:50.233041693 +0000 UTC m=+0.505387202 container init 9b7edc7a6ae843e78ac9085d231f2cfb1b39e9b029a6a1a6dd10ab370bfdac1a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:19:50.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 podman[44822]: 2024-09-16 08:19:50.236012502 +0000 UTC m=+0.508358009 container start 9b7edc7a6ae843e78ac9085d231f2cfb1b39e9b029a6a1a6dd10ab370bfdac1a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:19:50.611 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.259Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-16T08:19:50.611 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.259Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-16T08:19:50.611 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.259Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi193 (none))" 2024-09-16T08:19:50.611 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.259Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.259Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.261Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.261Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.262Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.262Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.264Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.264Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.57µs 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.264Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.264Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-16T08:19:50.612 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.264Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=16.393µs wal_replay_duration=253.613µs wbl_replay_duration=127ns total_replay_duration=287.796µs 2024-09-16T08:19:50.613 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.265Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T08:19:50.613 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.265Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-16T08:19:50.613 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.265Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T08:19:50.613 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.285Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.140041ms db_storage=667ns remote_storage=980ns web_handler=313ns query_engine=436ns scrape=461.754µs scrape_sd=72.373µs notify=752ns notify_sd=1.225µs rules=19.277368ms tracing=6.551µs 2024-09-16T08:19:50.613 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.285Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-16T08:19:50.613 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:19:50.285Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T08:19:50.613 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 bash[44822]: 9b7edc7a6ae843e78ac9085d231f2cfb1b39e9b029a6a1a6dd10ab370bfdac1a 2024-09-16T08:19:50.613 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:19:50 smithi193 systemd[1]: Started Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:19:50.770 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled grafana update... 2024-09-16T08:19:51.335 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-mon[25614]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:51.335 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.335 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.335 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.335 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T08:19:51.335 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.337 DEBUG:teuthology.orchestra.run.smithi193:grafana.a> sudo journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@grafana.a.service 2024-09-16T08:19:51.339 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T08:19:51.339 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T08:19:51.366 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[30018]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:51.366 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[23701]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T08:19:51.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' 2024-09-16T08:19:51.577 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:19:51.614 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: ignoring --setuser ceph since I am not root 2024-09-16T08:19:51.614 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: ignoring --setgroup ceph since I am not root 2024-09-16T08:19:51.614 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-mgr[27831]: -- 172.21.15.193:0/95664745 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f7026f6d00 con 0x55f7026cb400 2024-09-16T08:19:51.614 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:51.554+0000 7f53ba38f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T08:19:51.614 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:51.613+0000 7f53ba38f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T08:19:51.649 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ignoring --setuser ceph since I am not root 2024-09-16T08:19:51.649 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ignoring --setgroup ceph since I am not root 2024-09-16T08:19:51.650 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:51.585+0000 7efdb1536040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T08:19:51.650 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:51.649+0000 7efdb1536040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T08:19:51.870 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:51.768+0000 7f53ba38f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T08:19:51.870 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:51.838+0000 7f53ba38f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T08:19:51.905 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:51.800+0000 7efdb1536040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T08:19:51.906 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:51.869+0000 7efdb1536040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T08:19:52.136 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:52 smithi193 ceph-mon[25614]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:52.136 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:52 smithi193 ceph-mon[25614]: Saving service grafana spec with placement smithi193=a;count:1 2024-09-16T08:19:52.136 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:52 smithi193 ceph-mon[25614]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T08:19:52.136 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:52 smithi193 ceph-mon[25614]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T08:19:52.136 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:51.990+0000 7f53ba38f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:19:52.161 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:52.018+0000 7efdb1536040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:19:52.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-mon[30018]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:52.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-mon[30018]: Saving service grafana spec with placement smithi193=a;count:1 2024-09-16T08:19:52.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-mon[30018]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T08:19:52.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-mon[30018]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T08:19:52.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-mon[23701]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi193=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:19:52.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-mon[23701]: Saving service grafana spec with placement smithi193=a;count:1 2024-09-16T08:19:52.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-mon[23701]: from='mgr.14150 172.21.15.90:0/306791017' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T08:19:52.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-mon[23701]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-16T08:19:52.444 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:52 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:52.135+0000 7f53ba38f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T08:19:52.720 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:52.160+0000 7efdb1536040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T08:19:52.721 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:52.468+0000 7efdb1536040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T08:19:52.721 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:52.529+0000 7efdb1536040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T08:19:52.721 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:52 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:52.719+0000 7efdb1536040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T08:19:52.751 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:52 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:52.445+0000 7f53ba38f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T08:19:52.751 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:52 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:52.507+0000 7f53ba38f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T08:19:52.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:52 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:52.686+0000 7f53ba38f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T08:19:53.425 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-09-16T08:19:53.425 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQAp6udm5kAqGRAA75v85mavzQGvJceBVpOquA== 2024-09-16T08:19:53.711 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:53 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:53.642+0000 7efdb1536040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T08:19:53.711 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:53 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3560787566' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:19:53.712 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:53 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3560787566' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:19:53.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:53 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3560787566' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:19:53.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:53 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3560787566' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:19:53.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:53 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3560787566' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:19:53.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:53 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3560787566' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:19:53.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:53.552+0000 7f53ba38f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T08:19:53.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:53.618+0000 7f53ba38f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T08:19:54.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:53 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:53.711+0000 7efdb1536040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T08:19:54.119 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-16T08:19:54.119 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T08:19:54.119 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T08:19:54.198 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T08:19:54.414 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T08:19:54.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:54 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:54.330+0000 7f53ba38f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T08:19:54.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:54 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:54.469+0000 7f53ba38f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T08:19:54.684 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:54.428+0000 7efdb1536040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T08:19:54.684 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:54.543+0000 7efdb1536040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T08:19:54.805 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:54 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:54.627+0000 7f53ba38f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:19:54.806 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:54 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:54.690+0000 7f53ba38f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T08:19:54.955 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:54.683+0000 7efdb1536040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:19:54.955 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:54.744+0000 7efdb1536040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T08:19:54.956 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:54.888+0000 7efdb1536040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T08:19:55.135 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:54 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:54.845+0000 7f53ba38f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T08:19:55.135 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:54 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:54.912+0000 7f53ba38f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T08:19:55.135 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:54 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:54.973+0000 7f53ba38f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T08:19:55.230 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:54.955+0000 7efdb1536040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T08:19:55.231 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:55 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:55.014+0000 7efdb1536040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T08:19:55.231 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:55 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:55.167+0000 7efdb1536040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T08:19:55.415 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:55 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:55.134+0000 7f53ba38f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T08:19:55.415 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:55 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:55.205+0000 7f53ba38f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T08:19:55.415 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:55 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:55.272+0000 7f53ba38f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T08:19:55.415 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:55 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:55.355+0000 7f53ba38f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T08:19:55.415 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:55 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:55.414+0000 7f53ba38f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T08:19:55.493 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:55 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:55.230+0000 7efdb1536040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T08:19:55.493 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:55 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:55.295+0000 7efdb1536040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T08:19:55.493 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:55 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:55.377+0000 7efdb1536040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T08:19:55.493 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:55 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:55.434+0000 7efdb1536040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T08:19:55.493 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:55 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:55.492+0000 7efdb1536040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T08:19:55.727 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:55 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:55.475+0000 7f53ba38f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T08:19:56.360 INFO:teuthology.orchestra.run.smithi193.stdout:[client.1] 2024-09-16T08:19:56.361 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQAs6udm56VDFRAAdOgm96OU873x6sZ50Z+SKg== 2024-09-16T08:19:56.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:56.258+0000 7efdb1536040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T08:19:56.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:19:56] ENGINE Bus STARTING 2024-09-16T08:19:56.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: CherryPy Checker: 2024-09-16T08:19:56.563 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: The Application mounted at '' has an empty config. 2024-09-16T08:19:56.563 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:56.563 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:19:56] ENGINE Serving on http://:::9283 2024-09-16T08:19:56.563 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:19:56] ENGINE Bus STARTED 2024-09-16T08:19:56.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: Active manager daemon y restarted 2024-09-16T08:19:56.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: Activating manager daemon y 2024-09-16T08:19:56.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T08:19:56.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: mgrmap e17: y(active, starting, since 0.00646277s), standbys: x 2024-09-16T08:19:56.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:19:56.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: Manager daemon y is now available 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]': finished 2024-09-16T08:19:56.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: Active manager daemon y restarted 2024-09-16T08:19:56.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: Activating manager daemon y 2024-09-16T08:19:56.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T08:19:56.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: mgrmap e17: y(active, starting, since 0.00646277s), standbys: x 2024-09-16T08:19:56.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:19:56.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:19:56.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:19:56.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:19:56.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:19:56.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: Manager daemon y is now available 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:56.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:56.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:56.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]': finished 2024-09-16T08:19:56.634 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:56.315+0000 7f53ba38f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T08:19:56.634 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: [16/Sep/2024:08:19:56] ENGINE Bus STARTING 2024-09-16T08:19:56.635 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: CherryPy Checker: 2024-09-16T08:19:56.635 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: The Application mounted at '' has an empty config. 2024-09-16T08:19:56.635 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: 2024-09-16T08:19:56.635 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: [16/Sep/2024:08:19:56] ENGINE Serving on http://:::9283 2024-09-16T08:19:56.635 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[27827]: [16/Sep/2024:08:19:56] ENGINE Bus STARTED 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: Active manager daemon y restarted 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: Activating manager daemon y 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: mgrmap e17: y(active, starting, since 0.00646277s), standbys: x 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:19:56.636 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: Manager daemon y is now available 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:56.637 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:19:56.638 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]': finished 2024-09-16T08:19:57.593 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]': finished 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: Standby manager daemon x restarted 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: Standby manager daemon x started 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:19:57.594 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:19:57.595 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:19:57.595 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:19:57.595 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='client.? 172.21.15.193:0/838106176' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:19:57.595 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:19:57.595 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:19:57.595 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:57 smithi193 ceph-mon[25614]: mgrmap e18: y(active, since 1.01096s), standbys: x 2024-09-16T08:19:57.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]': finished 2024-09-16T08:19:57.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:57.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: Standby manager daemon x restarted 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: Standby manager daemon x started 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:19:57.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:19:57.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:19:57.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:19:57.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='client.? 172.21.15.193:0/838106176' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:19:57.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:19:57.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:19:57.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[23701]: mgrmap e18: y(active, since 1.01096s), standbys: x 2024-09-16T08:19:57.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]': finished 2024-09-16T08:19:57.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:19:57.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:19:57.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:19:57.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:19:57.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:19:57.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:57.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:19:57.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: Standby manager daemon x restarted 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: Standby manager daemon x started 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/1466803256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='client.? 172.21.15.193:0/838106176' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:19:57.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-mon[30018]: mgrmap e18: y(active, since 1.01096s), standbys: x 2024-09-16T08:19:57.761 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-09-16T08:19:57.761 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-16T08:19:57.761 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T08:19:57.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph config log 1 --format=json 2024-09-16T08:19:58.009 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:19:58.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:57.667+0000 7efd0f2b0640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:08:19:57] ENGINE Error in HTTPServer.serve 2024-09-16T08:19:58.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: Traceback (most recent call last): 2024-09-16T08:19:58.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:19:58.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self._connections.run(self.expiration_interval) 2024-09-16T08:19:58.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:19:58.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self._run(expiration_interval) 2024-09-16T08:19:58.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: s = self.context.wrap_socket( 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: return self.sslsocket_class._create( 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self.do_handshake() 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:19:58.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: self._sslobj.do_handshake() 2024-09-16T08:19:58.064 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:19:58.064 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.434 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.435 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.435 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.435 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.489 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: pgmap v3: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:58.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:19:59.523 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[30018]: [16/Sep/2024:08:19:57] ENGINE Bus STARTING 2024-09-16T08:19:59.524 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[30018]: Deploying daemon alertmanager.a on smithi090 2024-09-16T08:19:59.524 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[30018]: [16/Sep/2024:08:19:57] ENGINE Serving on https://172.21.15.90:7151 2024-09-16T08:19:59.524 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[30018]: [16/Sep/2024:08:19:57] ENGINE Error in HTTPServer.serve 2024-09-16T08:19:59.524 INFO:journalctl@ceph.mon.c.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:19:59.524 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:19:59.524 INFO:journalctl@ceph.mon.c.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:19:59.524 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:19:59.524 INFO:journalctl@ceph.mon.c.smithi090.stdout: self._run(expiration_interval) 2024-09-16T08:19:59.524 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: s = self.context.wrap_socket( 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: return self.sslsocket_class._create( 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: self.do_handshake() 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: self._sslobj.do_handshake() 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[30018]: [16/Sep/2024:08:19:57] ENGINE Serving on http://172.21.15.90:8765 2024-09-16T08:19:59.525 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[30018]: [16/Sep/2024:08:19:57] ENGINE Bus STARTED 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[30018]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[30018]: mgrmap e19: y(active, since 2s), standbys: x 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[23701]: [16/Sep/2024:08:19:57] ENGINE Bus STARTING 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[23701]: Deploying daemon alertmanager.a on smithi090 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[23701]: [16/Sep/2024:08:19:57] ENGINE Serving on https://172.21.15.90:7151 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[23701]: [16/Sep/2024:08:19:57] ENGINE Error in HTTPServer.serve 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:19:59.526 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._run(expiration_interval) 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: s = self.context.wrap_socket( 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.sslsocket_class._create( 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: self.do_handshake() 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._sslobj.do_handshake() 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:19:59.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[23701]: [16/Sep/2024:08:19:57] ENGINE Serving on http://172.21.15.90:8765 2024-09-16T08:19:59.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[23701]: [16/Sep/2024:08:19:57] ENGINE Bus STARTED 2024-09-16T08:19:59.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[23701]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:59.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-mon[23701]: mgrmap e19: y(active, since 2s), standbys: x 2024-09-16T08:19:59.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:59 smithi193 ceph-mon[25614]: [16/Sep/2024:08:19:57] ENGINE Bus STARTING 2024-09-16T08:19:59.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:59 smithi193 ceph-mon[25614]: Deploying daemon alertmanager.a on smithi090 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:59 smithi193 ceph-mon[25614]: [16/Sep/2024:08:19:57] ENGINE Serving on https://172.21.15.90:7151 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:59 smithi193 ceph-mon[25614]: [16/Sep/2024:08:19:57] ENGINE Error in HTTPServer.serve 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: Traceback (most recent call last): 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._run(expiration_interval) 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:19:59.757 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout: s = self.context.wrap_socket( 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout: return self.sslsocket_class._create( 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout: self.do_handshake() 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._sslobj.do_handshake() 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:59 smithi193 ceph-mon[25614]: [16/Sep/2024:08:19:57] ENGINE Serving on http://172.21.15.90:8765 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:59 smithi193 ceph-mon[25614]: [16/Sep/2024:08:19:57] ENGINE Bus STARTED 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:59 smithi193 ceph-mon[25614]: pgmap v4: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:19:59.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:19:59 smithi193 ceph-mon[25614]: mgrmap e19: y(active, since 2s), standbys: x 2024-09-16T08:20:00.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:19:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:19:59] "GET /metrics HTTP/1.1" 200 23676 "" "Prometheus/2.43.0" 2024-09-16T08:20:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:20:00.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:00 smithi193 ceph-mon[25614]: overall HEALTH_OK 2024-09-16T08:20:00.811 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:00 smithi090 ceph-mon[30018]: overall HEALTH_OK 2024-09-16T08:20:00.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:00 smithi090 ceph-mon[23701]: overall HEALTH_OK 2024-09-16T08:20:01.195 INFO:teuthology.orchestra.run.smithi090.stdout:[{"version":24,"timestamp":"2024-09-16T08:19:56.311558+0000","name":"","changes":[{"name":"osd/host:smithi090/osd_memory_target","previous_value":"3942650880"}]}] 2024-09-16T08:20:01.196 INFO:tasks.ceph_manager:config epoch is 24 2024-09-16T08:20:01.196 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T08:20:01.197 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T08:20:01.197 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T08:20:01.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:01 smithi090 ceph-mon[30018]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:01.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:01 smithi090 ceph-mon[30018]: mgrmap e20: y(active, since 4s), standbys: x 2024-09-16T08:20:01.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:01 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1110306939' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-16T08:20:01.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:01 smithi090 ceph-mon[23701]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:01.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:01 smithi090 ceph-mon[23701]: mgrmap e20: y(active, since 4s), standbys: x 2024-09-16T08:20:01.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:01 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1110306939' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-16T08:20:01.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:01 smithi193 ceph-mon[25614]: pgmap v5: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:01.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:01 smithi193 ceph-mon[25614]: mgrmap e20: y(active, since 4s), standbys: x 2024-09-16T08:20:01.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:01 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1110306939' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-16T08:20:02.214 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:03.595 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:03 smithi193 ceph-mon[25614]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:03.797 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:03 smithi090 ceph-mon[30018]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:03.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:03 smithi090 ceph-mon[23701]: pgmap v6: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:05.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:05 smithi090 ceph-mon[30018]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-16T08:20:05.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:05 smithi090 ceph-mon[23701]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-16T08:20:05.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:05 smithi193 ceph-mon[25614]: pgmap v7: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-09-16T08:20:06.239 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:20:06.497 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:06 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3714890029' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T08:20:06.497 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:06 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3714890029' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T08:20:06.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:06 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3714890029' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T08:20:07.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:07 smithi090 ceph-mon[30018]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T08:20:07.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:07 smithi090 ceph-mon[23701]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T08:20:07.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:07 smithi193 ceph-mon[25614]: pgmap v8: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T08:20:09.123 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":20,"flags":0,"active_gid":14457,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3507480917},{"type":"v1","addr":"172.21.15.90:6801","nonce":3507480917}]},"active_addr":"172.21.15.90:6801/3507480917","active_change":"2024-09-16T08:19:56.260839+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24328,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.90:8443/","prometheus":"http://172.21.15.90:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":62,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2381174981}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":4275095857}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":960941707}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2594198040}]}]} 2024-09-16T08:20:09.129 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T08:20:09.129 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T08:20:09.130 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T08:20:09.444 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-mon[30018]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T08:20:09.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-mon[23701]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T08:20:09.445 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 podman[51517]: 2024-09-16 08:20:09.445185755 +0000 UTC m=+0.215327410 container create 12a52cca346913162bdb9ccb03a4e6b5fbb0f6177b338dd0375a7cf56b25c2ec (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:20:09.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:09 smithi193 ceph-mon[25614]: pgmap v9: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T08:20:09.990 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:20:09] "GET /metrics HTTP/1.1" 200 23676 "" "Prometheus/2.43.0" 2024-09-16T08:20:09.990 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 podman[51517]: 2024-09-16 08:20:09.724685888 +0000 UTC m=+0.494827542 container init 12a52cca346913162bdb9ccb03a4e6b5fbb0f6177b338dd0375a7cf56b25c2ec (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 podman[51517]: 2024-09-16 08:20:09.727404821 +0000 UTC m=+0.497546478 container start 12a52cca346913162bdb9ccb03a4e6b5fbb0f6177b338dd0375a7cf56b25c2ec (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:09.739Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:09.739Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:09.740Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.90 port=9094 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:09.741Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:09.765Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:09.765Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:09.768Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:09.768Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 bash[51517]: 12a52cca346913162bdb9ccb03a4e6b5fbb0f6177b338dd0375a7cf56b25c2ec 2024-09-16T08:20:09.991 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:09 smithi090 systemd[1]: Started Ceph alertmanager.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:20:10.261 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:20:09] ENGINE Bus STOPPING 2024-09-16T08:20:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:10.561 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:20:10] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:20:10.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:20:10] ENGINE Bus STOPPED 2024-09-16T08:20:10.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:20:10] ENGINE Bus STARTING 2024-09-16T08:20:10.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:20:10] ENGINE Serving on http://:::9283 2024-09-16T08:20:10.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:20:10] ENGINE Bus STARTED 2024-09-16T08:20:11.188 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.188 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.188 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.188 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.188 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: Deploying daemon grafana.a on smithi193 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[30018]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T08:20:11.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:20:11.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:20:11.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: Deploying daemon grafana.a on smithi193 2024-09-16T08:20:11.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:10 smithi090 ceph-mon[23701]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:20:11.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T08:20:11.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:20:11.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:20:11.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:11.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: Deploying daemon grafana.a on smithi193 2024-09-16T08:20:11.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:10 smithi193 ceph-mon[25614]: pgmap v10: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:20:12.054 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:11 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:11.741Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000417925s 2024-09-16T08:20:12.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:12.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:12.811 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:12.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:12.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:12.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:13.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:13 smithi090 ceph-mon[23701]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:20:13.622 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:13 smithi090 ceph-mon[30018]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:20:13.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:13 smithi193 ceph-mon[25614]: pgmap v11: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:20:13.987 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:20:13.987 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":62,"fsid":"8b378aa4-7403-11ef-bceb-c7b262605968","created":"2024-09-16T08:14:17.250796+0000","modified":"2024-09-16T08:19:56.260683+0000","last_up_change":"2024-09-16T08:19:25.082991+0000","last_in_change":"2024-09-16T08:19:07.667092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":19,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T08:17:26.343055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d5e8461-ab0d-447a-9c03-fbf862340931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":10,"up_from":13,"up_thru":59,"down_at":11,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6803","nonce":2713629494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6805","nonce":2713629494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6809","nonce":2713629494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6807","nonce":2713629494}]},"public_addr":"172.21.15.90:6803/2713629494","cluster_addr":"172.21.15.90:6805/2713629494","heartbeat_back_addr":"172.21.15.90:6809/2713629494","heartbeat_front_addr":"172.21.15.90:6807/2713629494","state":["exists","up"]},{"osd":1,"uuid":"18d07a14-8eb1-484a-a16b-6be128901ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6811","nonce":3883828403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6813","nonce":3883828403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6817","nonce":3883828403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6815","nonce":3883828403}]},"public_addr":"172.21.15.90:6811/3883828403","cluster_addr":"172.21.15.90:6813/3883828403","heartbeat_back_addr":"172.21.15.90:6817/3883828403","heartbeat_front_addr":"172.21.15.90:6815/3883828403","state":["exists","up"]},{"osd":2,"uuid":"55d2c1c2-391b-404c-aa8e-4955fc69038f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6819","nonce":1303698361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6821","nonce":1303698361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6825","nonce":1303698361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6823","nonce":1303698361}]},"public_addr":"172.21.15.90:6819/1303698361","cluster_addr":"172.21.15.90:6821/1303698361","heartbeat_back_addr":"172.21.15.90:6825/1303698361","heartbeat_front_addr":"172.21.15.90:6823/1303698361","state":["exists","up"]},{"osd":3,"uuid":"68684ffb-a84b-4dd1-8b09-b779d0b4b930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6827","nonce":4238944051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6829","nonce":4238944051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6833","nonce":4238944051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6831","nonce":4238944051}]},"public_addr":"172.21.15.90:6827/4238944051","cluster_addr":"172.21.15.90:6829/4238944051","heartbeat_back_addr":"172.21.15.90:6833/4238944051","heartbeat_front_addr":"172.21.15.90:6831/4238944051","state":["exists","up"]},{"osd":4,"uuid":"4aeb8730-714e-4175-b4c4-f276993b0a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6801","nonce":2641807789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6803","nonce":2641807789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6807","nonce":2641807789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6805","nonce":2641807789}]},"public_addr":"172.21.15.193:6801/2641807789","cluster_addr":"172.21.15.193:6803/2641807789","heartbeat_back_addr":"172.21.15.193:6807/2641807789","heartbeat_front_addr":"172.21.15.193:6805/2641807789","state":["exists","up"]},{"osd":5,"uuid":"3192b17d-4647-49ec-9385-a997434b9a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6809","nonce":1961012311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6811","nonce":1961012311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6815","nonce":1961012311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6813","nonce":1961012311}]},"public_addr":"172.21.15.193:6809/1961012311","cluster_addr":"172.21.15.193:6811/1961012311","heartbeat_back_addr":"172.21.15.193:6815/1961012311","heartbeat_front_addr":"172.21.15.193:6813/1961012311","state":["exists","up"]},{"osd":6,"uuid":"3357150d-8a24-4f20-9778-d5d345a7ed94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6817","nonce":623377406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6819","nonce":623377406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6823","nonce":623377406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6821","nonce":623377406}]},"public_addr":"172.21.15.193:6817/623377406","cluster_addr":"172.21.15.193:6819/623377406","heartbeat_back_addr":"172.21.15.193:6823/623377406","heartbeat_front_addr":"172.21.15.193:6821/623377406","state":["exists","up"]},{"osd":7,"uuid":"2bb59f2b-d01f-4f5e-80d6-a4bec26999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6825","nonce":424442086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6827","nonce":424442086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6831","nonce":424442086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6829","nonce":424442086}]},"public_addr":"172.21.15.193:6825/424442086","cluster_addr":"172.21.15.193:6827/424442086","heartbeat_back_addr":"172.21.15.193:6831/424442086","heartbeat_front_addr":"172.21.15.193:6829/424442086","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:16:30.515585+0000","dead_epoch":10},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:17:00.317818+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:17:23.471718+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:17:47.493781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:18:12.442823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:18:35.677918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:18:59.853193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:19:23.825797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3621746675":"2024-09-17T08:19:56.260650+0000","172.21.15.90:0/1992058521":"2024-09-17T08:19:56.260650+0000","172.21.15.90:0/2522025803":"2024-09-17T08:14:38.353619+0000","172.21.15.90:0/2556824909":"2024-09-17T08:14:38.353619+0000","172.21.15.90:0/2766943845":"2024-09-17T08:14:38.353619+0000","172.21.15.90:6801/1569791063":"2024-09-17T08:19:56.260650+0000","172.21.15.90:6800/688775638":"2024-09-17T08:14:38.353619+0000","172.21.15.90:6801/688775638":"2024-09-17T08:14:38.353619+0000","172.21.15.90:0/1172167221":"2024-09-17T08:19:56.260650+0000","172.21.15.90:6800/1569791063":"2024-09-17T08:19:56.260650+0000","172.21.15.90:0/3342887308":"2024-09-17T08:15:10.045545+0000","172.21.15.90:0/3754108410":"2024-09-17T08:15:10.045545+0000","172.21.15.90:6800/2155974574":"2024-09-17T08:15:10.045545+0000","172.21.15.90:0/1806888503":"2024-09-17T08:19:56.260650+0000","172.21.15.90:0/392352616":"2024-09-17T08:15:10.045545+0000","172.21.15.90:6801/2155974574":"2024-09-17T08:15:10.045545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T08:20:14.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:14 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/66620232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:20:14.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:14 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/66620232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:20:14.719 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T08:20:14.719 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T08:20:14.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:14 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/66620232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:20:15.484 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:15.607 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:15 smithi090 ceph-mon[30018]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:20:15.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:15 smithi090 ceph-mon[23701]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:20:15.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:15 smithi193 ceph-mon[25614]: pgmap v12: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:20:17.611 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:17 smithi090 ceph-mon[30018]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:17.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:17 smithi090 ceph-mon[23701]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:17.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:17 smithi193 ceph-mon[25614]: pgmap v13: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:17.801 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:20:17.801 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":62,"fsid":"8b378aa4-7403-11ef-bceb-c7b262605968","created":"2024-09-16T08:14:17.250796+0000","modified":"2024-09-16T08:19:56.260683+0000","last_up_change":"2024-09-16T08:19:25.082991+0000","last_in_change":"2024-09-16T08:19:07.667092+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":19,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T08:17:26.343055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d5e8461-ab0d-447a-9c03-fbf862340931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":10,"up_from":13,"up_thru":59,"down_at":11,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6803","nonce":2713629494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6805","nonce":2713629494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6809","nonce":2713629494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2713629494},{"type":"v1","addr":"172.21.15.90:6807","nonce":2713629494}]},"public_addr":"172.21.15.90:6803/2713629494","cluster_addr":"172.21.15.90:6805/2713629494","heartbeat_back_addr":"172.21.15.90:6809/2713629494","heartbeat_front_addr":"172.21.15.90:6807/2713629494","state":["exists","up"]},{"osd":1,"uuid":"18d07a14-8eb1-484a-a16b-6be128901ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6811","nonce":3883828403}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6813","nonce":3883828403}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6817","nonce":3883828403}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3883828403},{"type":"v1","addr":"172.21.15.90:6815","nonce":3883828403}]},"public_addr":"172.21.15.90:6811/3883828403","cluster_addr":"172.21.15.90:6813/3883828403","heartbeat_back_addr":"172.21.15.90:6817/3883828403","heartbeat_front_addr":"172.21.15.90:6815/3883828403","state":["exists","up"]},{"osd":2,"uuid":"55d2c1c2-391b-404c-aa8e-4955fc69038f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6819","nonce":1303698361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6821","nonce":1303698361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6825","nonce":1303698361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1303698361},{"type":"v1","addr":"172.21.15.90:6823","nonce":1303698361}]},"public_addr":"172.21.15.90:6819/1303698361","cluster_addr":"172.21.15.90:6821/1303698361","heartbeat_back_addr":"172.21.15.90:6825/1303698361","heartbeat_front_addr":"172.21.15.90:6823/1303698361","state":["exists","up"]},{"osd":3,"uuid":"68684ffb-a84b-4dd1-8b09-b779d0b4b930","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":28,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6827","nonce":4238944051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6829","nonce":4238944051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6833","nonce":4238944051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":4238944051},{"type":"v1","addr":"172.21.15.90:6831","nonce":4238944051}]},"public_addr":"172.21.15.90:6827/4238944051","cluster_addr":"172.21.15.90:6829/4238944051","heartbeat_back_addr":"172.21.15.90:6833/4238944051","heartbeat_front_addr":"172.21.15.90:6831/4238944051","state":["exists","up"]},{"osd":4,"uuid":"4aeb8730-714e-4175-b4c4-f276993b0a16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6801","nonce":2641807789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6803","nonce":2641807789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6807","nonce":2641807789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":2641807789},{"type":"v1","addr":"172.21.15.193:6805","nonce":2641807789}]},"public_addr":"172.21.15.193:6801/2641807789","cluster_addr":"172.21.15.193:6803/2641807789","heartbeat_back_addr":"172.21.15.193:6807/2641807789","heartbeat_front_addr":"172.21.15.193:6805/2641807789","state":["exists","up"]},{"osd":5,"uuid":"3192b17d-4647-49ec-9385-a997434b9a64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6809","nonce":1961012311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6811","nonce":1961012311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6815","nonce":1961012311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":1961012311},{"type":"v1","addr":"172.21.15.193:6813","nonce":1961012311}]},"public_addr":"172.21.15.193:6809/1961012311","cluster_addr":"172.21.15.193:6811/1961012311","heartbeat_back_addr":"172.21.15.193:6815/1961012311","heartbeat_front_addr":"172.21.15.193:6813/1961012311","state":["exists","up"]},{"osd":6,"uuid":"3357150d-8a24-4f20-9778-d5d345a7ed94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6817","nonce":623377406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6819","nonce":623377406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6823","nonce":623377406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":623377406},{"type":"v1","addr":"172.21.15.193:6821","nonce":623377406}]},"public_addr":"172.21.15.193:6817/623377406","cluster_addr":"172.21.15.193:6819/623377406","heartbeat_back_addr":"172.21.15.193:6823/623377406","heartbeat_front_addr":"172.21.15.193:6821/623377406","state":["exists","up"]},{"osd":7,"uuid":"2bb59f2b-d01f-4f5e-80d6-a4bec26999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6825","nonce":424442086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6827","nonce":424442086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6831","nonce":424442086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":424442086},{"type":"v1","addr":"172.21.15.193:6829","nonce":424442086}]},"public_addr":"172.21.15.193:6825/424442086","cluster_addr":"172.21.15.193:6827/424442086","heartbeat_back_addr":"172.21.15.193:6831/424442086","heartbeat_front_addr":"172.21.15.193:6829/424442086","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:16:30.515585+0000","dead_epoch":10},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:17:00.317818+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:17:23.471718+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:17:47.493781+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:18:12.442823+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:18:35.677918+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:18:59.853193+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:19:23.825797+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3621746675":"2024-09-17T08:19:56.260650+0000","172.21.15.90:0/1992058521":"2024-09-17T08:19:56.260650+0000","172.21.15.90:0/2522025803":"2024-09-17T08:14:38.353619+0000","172.21.15.90:0/2556824909":"2024-09-17T08:14:38.353619+0000","172.21.15.90:0/2766943845":"2024-09-17T08:14:38.353619+0000","172.21.15.90:6801/1569791063":"2024-09-17T08:19:56.260650+0000","172.21.15.90:6800/688775638":"2024-09-17T08:14:38.353619+0000","172.21.15.90:6801/688775638":"2024-09-17T08:14:38.353619+0000","172.21.15.90:0/1172167221":"2024-09-17T08:19:56.260650+0000","172.21.15.90:6800/1569791063":"2024-09-17T08:19:56.260650+0000","172.21.15.90:0/3342887308":"2024-09-17T08:15:10.045545+0000","172.21.15.90:0/3754108410":"2024-09-17T08:15:10.045545+0000","172.21.15.90:6800/2155974574":"2024-09-17T08:15:10.045545+0000","172.21.15.90:0/1806888503":"2024-09-17T08:19:56.260650+0000","172.21.15.90:0/392352616":"2024-09-17T08:15:10.045545+0000","172.21.15.90:6801/2155974574":"2024-09-17T08:15:10.045545+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T08:20:18.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:18 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1771742782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:20:18.506 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T08:20:18.506 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T08:20:18.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T08:20:18.507 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T08:20:18.508 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T08:20:18.508 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T08:20:18.509 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T08:20:18.509 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T08:20:18.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:18 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1771742782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:20:18.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:18 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1771742782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:20:18.962 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:18.962 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:18.963 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:18.968 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:18.971 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:18.979 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:18.982 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:18.997 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:19.373 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:19 smithi090 ceph-mon[30018]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:19.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:19 smithi090 ceph-mon[23701]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:19 smithi193 ceph-mon[25614]: pgmap v14: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:19.961 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:19 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:20:19] "GET /metrics HTTP/1.1" 200 26203 "" "Prometheus/2.43.0" 2024-09-16T08:20:19.962 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:20:19 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:20:19.743Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002317642s 2024-09-16T08:20:21.222 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 systemd[1]: Started Ceph grafana.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:20:21.472 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-mon[25614]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:21.473 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.473 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.473 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.473 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.473 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.474 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222548847Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-16T08:20:21.474 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222801209Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-16T08:20:21.474 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222871208Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222883306Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222892734Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222902443Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222911704Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222921821Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222931545Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222940971Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222950206Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.22296636Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222976377Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=settings t=2024-09-16T08:20:21.222986086Z level=info msg="App mode production" 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=sqlstore t=2024-09-16T08:20:21.223050923Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-16T08:20:21.475 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=sqlstore t=2024-09-16T08:20:21.223075602Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.22436986Z level=info msg="Starting DB migrations" 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.224479769Z level=info msg="Executing migration" id="create migration_log table" 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.225073831Z level=info msg="Executing migration" id="create user table" 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.225512806Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.226398438Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.227226031Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.228043536Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.228814848Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.229679123Z level=info msg="Executing migration" id="create user table v2" 2024-09-16T08:20:21.476 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.2305892Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-16T08:20:21.477 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.231432083Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-16T08:20:21.477 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.232323952Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-16T08:20:21.477 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.233170648Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-16T08:20:21.477 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.235933637Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-16T08:20:21.477 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.238614468Z level=info msg="Executing migration" id="Update user table charset" 2024-09-16T08:20:21.478 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.238951776Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-16T08:20:21.478 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.241748759Z level=info msg="Executing migration" id="Add missing user data" 2024-09-16T08:20:21.478 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.244533001Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-16T08:20:21.478 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.247139984Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-16T08:20:21.478 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.247609465Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-16T08:20:21.478 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.250353775Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-16T08:20:21.479 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.251693397Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-16T08:20:21.479 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.253120516Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-16T08:20:21.479 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.255870449Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-16T08:20:21.479 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.258666437Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-16T08:20:21.479 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.261374594Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-16T08:20:21.479 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.264085845Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-16T08:20:21.479 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.266727668Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-16T08:20:21.480 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.269539609Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-16T08:20:21.480 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.272246708Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-16T08:20:21.480 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.274877064Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-16T08:20:21.480 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.27770455Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-16T08:20:21.480 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.28021578Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-16T08:20:21.480 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.282843843Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-16T08:20:21.480 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.285662316Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-16T08:20:21.481 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.288123148Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-16T08:20:21.481 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.290954662Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-16T08:20:21.481 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.293540276Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-16T08:20:21.481 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.296045365Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-16T08:20:21.481 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.298640421Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-16T08:20:21.481 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.301465103Z level=info msg="Executing migration" id="create star table" 2024-09-16T08:20:21.482 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.301866837Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-16T08:20:21.482 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.30472425Z level=info msg="Executing migration" id="create org table v1" 2024-09-16T08:20:21.482 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.307244613Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-16T08:20:21.482 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.310259904Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-16T08:20:21.482 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.31284792Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-16T08:20:21.482 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.315675271Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-16T08:20:21.483 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.318241816Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-16T08:20:21.483 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.318750453Z level=info msg="Executing migration" id="Update org table charset" 2024-09-16T08:20:21.483 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.319544459Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-16T08:20:21.483 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.320362153Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-16T08:20:21.483 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.321129135Z level=info msg="Executing migration" id="create dashboard table" 2024-09-16T08:20:21.483 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.321911785Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-16T08:20:21.484 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.322725968Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-16T08:20:21.484 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.323663655Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-16T08:20:21.484 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.324377637Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-16T08:20:21.484 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.325109864Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-16T08:20:21.484 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.32600973Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-16T08:20:21.484 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.327768882Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-16T08:20:21.485 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.328536589Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-16T08:20:21.485 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.329340089Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-16T08:20:21.485 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.330120704Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-16T08:20:21.485 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.33107189Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-16T08:20:21.485 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.331797898Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-16T08:20:21.486 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.332643965Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-16T08:20:21.486 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.333433056Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-16T08:20:21.486 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.334390292Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-16T08:20:21.486 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.335113729Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-16T08:20:21.486 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.336002515Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-16T08:20:21.486 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.336814902Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-16T08:20:21.487 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.339363413Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-16T08:20:21.487 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.342112042Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-16T08:20:21.487 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.344837671Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-16T08:20:21.487 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.34745239Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-16T08:20:21.487 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.347979591Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-16T08:20:21.487 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.350160652Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-16T08:20:21.488 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.35261366Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-16T08:20:21.488 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.355404513Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-16T08:20:21.488 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.356135447Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-16T08:20:21.488 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.35657181Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-16T08:20:21.490 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.357370283Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-16T08:20:21.490 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.357695967Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-16T08:20:21.490 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.358162732Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-16T08:20:21.490 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.358986757Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-16T08:20:21.490 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.359888195Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-16T08:20:21.491 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.360703774Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-16T08:20:21.491 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.361491478Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-16T08:20:21.491 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.362358067Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-16T08:20:21.491 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.362723156Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-16T08:20:21.491 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.363624303Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-16T08:20:21.492 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.364395922Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-16T08:20:21.492 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.3648187Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-16T08:20:21.492 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.365632076Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-16T08:20:21.492 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.365992959Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-16T08:20:21.492 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.366783059Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-16T08:20:21.493 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.367749653Z level=info msg="Executing migration" id="create data_source table" 2024-09-16T08:20:21.493 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.368465415Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-16T08:20:21.493 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.368897338Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-16T08:20:21.493 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.369325111Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-16T08:20:21.494 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.370142244Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-16T08:20:21.494 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.370552232Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-16T08:20:21.494 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.371592876Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-16T08:20:21.494 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.372014804Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-16T08:20:21.494 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.372847953Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-16T08:20:21.495 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.37370267Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-16T08:20:21.495 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.374611621Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-16T08:20:21.495 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.375425347Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-16T08:20:21.495 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.375947776Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-16T08:20:21.495 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.376891907Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-16T08:20:21.495 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.377669837Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-16T08:20:21.496 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.378518779Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-16T08:20:21.496 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.378857186Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-16T08:20:21.496 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.379803002Z level=info msg="Executing migration" id="Add uid column" 2024-09-16T08:20:21.496 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.380527209Z level=info msg="Executing migration" id="Update uid value" 2024-09-16T08:20:21.496 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.383362738Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-16T08:20:21.497 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.383809257Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-16T08:20:21.497 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.386775082Z level=info msg="Executing migration" id="create api_key table" 2024-09-16T08:20:21.497 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.389472954Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-16T08:20:21.497 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.39219798Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-16T08:20:21.497 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.392613616Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-16T08:20:21.497 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.395633118Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-16T08:20:21.498 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.398141169Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-16T08:20:21.498 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.40088397Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-16T08:20:21.498 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.403522374Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-16T08:20:21.499 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.406276898Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-16T08:20:21.499 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.409076365Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-16T08:20:21.499 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.4117907Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-16T08:20:21.499 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.414673609Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-16T08:20:21.499 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.417532413Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-16T08:20:21.499 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.420171403Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-16T08:20:21.500 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.420636217Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-16T08:20:21.500 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.423483122Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-16T08:20:21.500 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.424176648Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-16T08:20:21.500 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.426395491Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-16T08:20:21.500 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.429013759Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-16T08:20:21.500 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.431705408Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-16T08:20:21.501 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.434486197Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-16T08:20:21.501 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.437019052Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-16T08:20:21.501 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.439719717Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-16T08:20:21.501 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.442290623Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-16T08:20:21.501 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.444865166Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-16T08:20:21.502 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.447550925Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-16T08:20:21.502 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.450132762Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-16T08:20:21.502 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.452657997Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-16T08:20:21.502 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.455463444Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-16T08:20:21.502 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.45811361Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-16T08:20:21.502 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.460936107Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-16T08:20:21.503 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.463515589Z level=info msg="Executing migration" id="create quota table v1" 2024-09-16T08:20:21.503 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.466236232Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-16T08:20:21.503 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.469067116Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-16T08:20:21.503 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.469736555Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-16T08:20:21.503 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.4701477Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-16T08:20:21.503 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.470955587Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-16T08:20:21.504 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.471801817Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-16T08:20:21.504 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.472535082Z level=info msg="Executing migration" id="create session table" 2024-09-16T08:20:21.504 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.473000585Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-16T08:20:21.504 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.473779142Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-16T08:20:21.504 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.474120972Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-16T08:20:21.505 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.474535272Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-16T08:20:21.505 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.475375108Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-16T08:20:21.505 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.476129159Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-16T08:20:21.505 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.476443123Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-16T08:20:21.505 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.476735549Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-16T08:20:21.505 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.47705619Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-16T08:20:21.506 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.477606023Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-16T08:20:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[23701]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.601 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[30018]: pgmap v15: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:21.602 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.602 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.602 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.602 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.602 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:21.724 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.47805477Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-16T08:20:21.724 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.478655439Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-16T08:20:21.724 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.478930914Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-16T08:20:21.724 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.47951857Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-16T08:20:21.724 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.480370211Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-16T08:20:21.724 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.481400548Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-16T08:20:21.724 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.481837168Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-16T08:20:21.724 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.482831395Z level=info msg="Executing migration" id="create alert table v1" 2024-09-16T08:20:21.724 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.48354642Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.484393644Z level=info msg="Executing migration" id="add index alert state" 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.484786852Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.48519935Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.485559661Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.485979401Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.48636826Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.487769752Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.488175521Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.488607108Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-16T08:20:21.725 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.488894892Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-16T08:20:21.726 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.489251864Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.489641298Z level=info msg="Executing migration" id="Add column is_default" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.490274562Z level=info msg="Executing migration" id="Add column frequency" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.49091309Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.491596502Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.492250795Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.492680169Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.492931465Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.493223473Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.49361962Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.494046045Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.494443176Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-16T08:20:21.727 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.494801774Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.495262379Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.495900608Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.496587404Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.496863325Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.497273926Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.497661926Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.498283829Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.499082745Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.499935908Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.500750484Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.501513423Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.501923141Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-16T08:20:21.728 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.502382477Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.503174541Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.504038845Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.504882146Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.505691034Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.506424059Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.507453679Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.50802908Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.509020378Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.50985245Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.510683719Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.511529015Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-16T08:20:21.729 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.514238595Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.515065125Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.517846652Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.520662238Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.52121008Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.52392247Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.526688125Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.529268689Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.529731797Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.532295441Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.535007641Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.537725855Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.538191458Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-16T08:20:21.730 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.540815768Z level=info msg="Executing migration" id="Move region to single row" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.543463718Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.546176807Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.546631688Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.54943294Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.552206987Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.554843355Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.555324497Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.558152216Z level=info msg="Executing migration" id="create test_data table" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.560574547Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.56319677Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.563632002Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.566343473Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-16T08:20:21.731 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.569111971Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.571684759Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.5720497Z level=info msg="Executing migration" id="create team table" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.574907331Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.577649465Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.580437812Z level=info msg="Executing migration" id="create team member table" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.581040422Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.583756278Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.586235436Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.589079533Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.591534153Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.592350664Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-16T08:20:21.732 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.59317467Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.594088134Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.594648452Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.59635544Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.596785865Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.597747261Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.598339864Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.59915187Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.600070739Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.600739446Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.60156003Z level=info msg="Executing migration" id="create tag table" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.602053744Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-16T08:20:21.733 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.602472157Z level=info msg="Executing migration" id="create login attempt table" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.603353764Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.604143563Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.605075776Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.607089378Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.608081113Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.608740955Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.609641003Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.610078743Z level=info msg="Executing migration" id="create user auth table" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.610463435Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.611386825Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.612134035Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.613172974Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-16T08:20:21.734 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.614009502Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.614852987Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.615778139Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.616617971Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.617493842Z level=info msg="Executing migration" id="create server_lock table" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.618366462Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.619322942Z level=info msg="Executing migration" id="create user auth token table" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.620075931Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.620949647Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.623757712Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.626578289Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.629284731Z level=info msg="Executing migration" id="create cache_data table" 2024-09-16T08:20:21.735 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.631925642Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.634720093Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.637477636Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.640086207Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.64282043Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.645535079Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.648421158Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.651171559Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.65392029Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.656648291Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.65933797Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.661920121Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-16T08:20:21.736 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.664754768Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-16T08:20:21.737 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.667518425Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-16T08:20:21.737 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.670180428Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-16T08:20:21.737 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.672878345Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-16T08:20:21.737 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.675457918Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-16T08:20:21.737 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.677982228Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-16T08:20:21.737 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.680620655Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-16T08:20:21.737 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.683213684Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.685893426Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.688454565Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.691096289Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.693665353Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.6965109Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.699083332Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.701690171Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.70437261Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.707831147Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.711168836Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.713876758Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-16T08:20:21.738 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.716511907Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.717396134Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.718202358Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.719004061Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.719828095Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.720586902Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.721420032Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.722258693Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.723049228Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.723927253Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.724426878Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.724875572Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.725724093Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-16T08:20:21.739 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.726553345Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-16T08:20:21.740 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.727005095Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-16T08:20:21.740 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.727823416Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-16T08:20:21.998 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.728648329Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-16T08:20:21.998 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.728990204Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-16T08:20:21.998 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.72944218Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-16T08:20:21.998 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.730265996Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-16T08:20:21.998 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.730756853Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-16T08:20:21.998 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.731105645Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.732120208Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.733159302Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.735999011Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.738814359Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.741557683Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.744257127Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.747011524Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.749819926Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.752486845Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.755274217Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-16T08:20:21.999 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.758016821Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-16T08:20:22.000 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.760929273Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-16T08:20:22.000 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.763741484Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-16T08:20:22.000 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.766434396Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-16T08:20:22.000 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.769186753Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.771980569Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.774755627Z level=info msg="Executing migration" id="create alert_image table" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.777538216Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.780223266Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.782999589Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.785699808Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.78626391Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.786372407Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.786746296Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.789435883Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.792170896Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-16T08:20:22.001 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.792340901Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.792518128Z level=info msg="alerts found to migrate" alerts=0 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.794628715Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.79514333Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.797853824Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.800586935Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.803149344Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.803652898Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.806101629Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.806574354Z level=info msg="Executing migration" id="create data_keys table" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.808923818Z level=info msg="Executing migration" id="create secrets table" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.811731856Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.816228288Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-16T08:20:22.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.818976406Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.821728711Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.826385277Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.830926969Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.831775287Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.832526913Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.833370355Z level=info msg="Executing migration" id="create permission table" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.833909963Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.834708359Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.835490094Z level=info msg="Executing migration" id="create role table" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.836262996Z level=info msg="Executing migration" id="add column display_name" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.837381183Z level=info msg="Executing migration" id="add column group_name" 2024-09-16T08:20:22.003 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.83838015Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.839154682Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.839963204Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.840445042Z level=info msg="Executing migration" id="create team role table" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.841224644Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.842069312Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.842852788Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.843324453Z level=info msg="Executing migration" id="create user role table" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.844162008Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.845028173Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.845874959Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.846371896Z level=info msg="Executing migration" id="create builtin role table" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.847218242Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-16T08:20:22.004 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.848065079Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.848911685Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.849926663Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.850450066Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.851237368Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.851766039Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.852219639Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.852610429Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.85308343Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.854092447Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.854650005Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.855280216Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.855687333Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-16T08:20:22.005 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.856283407Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.856980459Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.857488033Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.857849512Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.858327482Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.858754792Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.85921123Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.860224656Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.860570185Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.861034582Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-16T08:20:22.006 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.861505495Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-16T08:20:22.007 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.862010464Z level=info msg="Executing migration" id="add correlation config column" 2024-09-16T08:20:22.007 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.863572098Z level=info msg="Executing migration" id="create entity_events table" 2024-09-16T08:20:22.007 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.864208029Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-16T08:20:22.007 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.864790025Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T08:20:22.007 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.864911581Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T08:20:22.007 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.865316744Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.865421452Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.865733499Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.866188225Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.866628084Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.867093886Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.867573202Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.867999723Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.868488273Z level=info msg="Executing migration" id="Drop public config table" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.868920204Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.869376636Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.869834389Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.870339046Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-16T08:20:22.008 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.870815632Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.873924703Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.875037767Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.876048233Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.87638733Z level=info msg="Executing migration" id="add share column" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.877443042Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.87804198Z level=info msg="Executing migration" id="create file table" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.878513763Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.878967903Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.879488739Z level=info msg="Executing migration" id="create file_meta table" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.879902614Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.882676838Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.885516368Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-16T08:20:22.009 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.888234838Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.888606902Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.891395334Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.89319461Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.8940455Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.896789753Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.89729297Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.898046818Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.898490592Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.899656558Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.901245057Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.901823869Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.914061034Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.914788437Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.915564055Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-16T08:20:22.010 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.920578531Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-16T08:20:22.011 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.921324012Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-16T08:20:22.011 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.922113715Z level=info msg="Executing migration" id="create folder table" 2024-09-16T08:20:22.011 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.922904344Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-16T08:20:22.011 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.923418825Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-16T08:20:22.011 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.924212988Z level=info msg="Executing migration" id="Update folder title length" 2024-09-16T08:20:22.011 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.925063176Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-16T08:20:22.011 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=migrator t=2024-09-16T08:20:21.925802018Z level=info msg="migrations completed" performed=485 skipped=0 duration=701.351545ms 2024-09-16T08:20:22.011 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=sqlstore t=2024-09-16T08:20:21.926274655Z level=info msg="Created default organization" 2024-09-16T08:20:22.011 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=plugin.loader t=2024-09-16T08:20:21.998006832Z level=info msg="Plugin registered" pluginID=input 2024-09-16T08:20:22.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=plugin.loader t=2024-09-16T08:20:22.032346707Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-16T08:20:22.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=plugin.loader t=2024-09-16T08:20:22.032399294Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-16T08:20:22.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=secrets t=2024-09-16T08:20:22.032441203Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-16T08:20:22.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=query_data t=2024-09-16T08:20:22.034472332Z level=info msg="Query Service initialization" 2024-09-16T08:20:22.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=live.push_http t=2024-09-16T08:20:22.037107051Z level=info msg="Live Push Gateway initialization" 2024-09-16T08:20:22.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:22 smithi090 ceph-mon[23701]: Deploying daemon node-exporter.a on smithi090 2024-09-16T08:20:22.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:22 smithi090 ceph-mon[30018]: Deploying daemon node-exporter.a on smithi090 2024-09-16T08:20:22.634 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-mon[25614]: Deploying daemon node-exporter.a on smithi090 2024-09-16T08:20:22.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=infra.usagestats.collector t=2024-09-16T08:20:22.830051482Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-16T08:20:22.885 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=provisioning.datasources t=2024-09-16T08:20:22.886019581Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-16T08:20:23.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=provisioning.datasources t=2024-09-16T08:20:22.917851043Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-16T08:20:23.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=provisioning.alerting t=2024-09-16T08:20:22.940031969Z level=info msg="starting to provision alerting" 2024-09-16T08:20:23.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=provisioning.alerting t=2024-09-16T08:20:22.940062846Z level=info msg="finished to provision alerting" 2024-09-16T08:20:23.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=grafanaStorageLogger t=2024-09-16T08:20:22.94020179Z level=info msg="storage starting" 2024-09-16T08:20:23.252 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=ngalert.state.manager t=2024-09-16T08:20:22.940368697Z level=info msg="Warming state cache for startup" 2024-09-16T08:20:23.253 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=http.server t=2024-09-16T08:20:22.943800357Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-16T08:20:23.253 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=ngalert.state.manager t=2024-09-16T08:20:22.950523918Z level=info msg="State cache has been initialized" states=0 duration=10.153809ms 2024-09-16T08:20:23.253 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=ticker t=2024-09-16T08:20:22.950597297Z level=info msg=starting first_tick=2024-09-16T08:20:30Z 2024-09-16T08:20:23.253 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=ngalert.multiorg.alertmanager t=2024-09-16T08:20:22.950614844Z level=info msg="starting MultiOrg Alertmanager" 2024-09-16T08:20:23.253 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=sqlstore.transactions t=2024-09-16T08:20:22.983756038Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T08:20:23.253 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:23 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=sqlstore.transactions t=2024-09-16T08:20:23.023988607Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T08:20:23.253 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:20:23 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a[47033]: logger=sqlstore.transactions t=2024-09-16T08:20:23.034383125Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T08:20:23.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:23 smithi193 ceph-mon[25614]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:23.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:23 smithi090 ceph-mon[30018]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:23.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:23 smithi090 ceph-mon[23701]: pgmap v16: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:24.828 INFO:teuthology.orchestra.run.smithi090.stdout:176093659159 2024-09-16T08:20:24.828 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T08:20:25.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:25 smithi090 ceph-mon[23701]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:25.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:25 smithi090 ceph-mon[30018]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:25.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:25 smithi193 ceph-mon[25614]: pgmap v17: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:26.631 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:26 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:26.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:26 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:26.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:26 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:27.716 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:27 smithi090 ceph-mon[30018]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:27.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:27 smithi090 ceph-mon[23701]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:27.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:27 smithi193 ceph-mon[25614]: pgmap v18: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:29.312 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:28 smithi090 bash[53509]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-16T08:20:29.719 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:29 smithi090 ceph-mon[30018]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:29.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:29 smithi090 ceph-mon[23701]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:29.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:29 smithi193 ceph-mon[25614]: pgmap v19: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:30.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:29 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:20:29] "GET /metrics HTTP/1.1" 200 26198 "" "Prometheus/2.43.0" 2024-09-16T08:20:30.280 INFO:teuthology.orchestra.run.smithi090.stdout:55834574892 2024-09-16T08:20:30.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T08:20:31.356 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:31 smithi090 bash[53509]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-16T08:20:31.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:31 smithi193 ceph-mon[25614]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:31.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:31 smithi090 ceph-mon[23701]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:31.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:31 smithi090 ceph-mon[30018]: pgmap v20: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:31.812 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:31 smithi090 bash[53509]: Writing manifest to image destination 2024-09-16T08:20:32.312 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:32 smithi090 podman[53509]: 2024-09-16 08:20:32.016417945 +0000 UTC m=+4.003809770 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-16T08:20:32.810 INFO:teuthology.orchestra.run.smithi090.stdout:68719476779 2024-09-16T08:20:32.811 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T08:20:33.606 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:33 smithi090 ceph-mon[30018]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:33.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:33 smithi090 ceph-mon[23701]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:33.683 INFO:teuthology.orchestra.run.smithi090.stdout:141733920798 2024-09-16T08:20:33.683 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T08:20:33.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:33 smithi193 ceph-mon[25614]: pgmap v21: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:35 smithi193 ceph-mon[25614]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:35.798 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:35 smithi090 ceph-mon[30018]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:35.799 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:35 smithi090 ceph-mon[23701]: pgmap v22: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:35.821 INFO:teuthology.orchestra.run.smithi090.stdout:90194313256 2024-09-16T08:20:35.822 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T08:20:36.062 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:35 smithi090 podman[53509]: 2024-09-16 08:20:35.940679598 +0000 UTC m=+7.928071415 container create 4a6ade533b17f416fdd3e3ff65c51810a414edb8ddfc91f7f748cecf266d7e15 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T08:20:36.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 podman[53509]: 2024-09-16 08:20:36.15144349 +0000 UTC m=+8.138835307 container init 4a6ade533b17f416fdd3e3ff65c51810a414edb8ddfc91f7f748cecf266d7e15 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T08:20:36.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 podman[53509]: 2024-09-16 08:20:36.154449994 +0000 UTC m=+8.141841829 container start 4a6ade533b17f416fdd3e3ff65c51810a414edb8ddfc91f7f748cecf266d7e15 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T08:20:36.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.158Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-16T08:20:36.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.158Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-16T08:20:36.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.158Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T08:20:36.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.158Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T08:20:36.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T08:20:36.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T08:20:36.563 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T08:20:36.563 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T08:20:36.563 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T08:20:36.563 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T08:20:36.563 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T08:20:36.563 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T08:20:36.563 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T08:20:36.563 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T08:20:36.564 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T08:20:36.564 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T08:20:36.564 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T08:20:36.564 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T08:20:36.564 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T08:20:36.564 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T08:20:36.564 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T08:20:36.564 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T08:20:36.564 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T08:20:36.565 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T08:20:36.566 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T08:20:36.567 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T08:20:36.567 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.159Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T08:20:36.567 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.160Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-16T08:20:36.567 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a[54042]: ts=2024-09-16T08:20:36.160Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T08:20:36.567 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 bash[53509]: 4a6ade533b17f416fdd3e3ff65c51810a414edb8ddfc91f7f748cecf266d7e15 2024-09-16T08:20:36.567 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:20:36 smithi090 systemd[1]: Started Ceph node-exporter.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:20:37.225 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:37 smithi193 systemd[1]: Starting Ceph node-exporter.b for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:20:37.501 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:37 smithi193 bash[47944]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-16T08:20:37.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:37 smithi193 ceph-mon[25614]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:37.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:37 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:37.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:37 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:37.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:37 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:37.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:37 smithi193 ceph-mon[25614]: Deploying daemon node-exporter.b on smithi193 2024-09-16T08:20:37.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[30018]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:37.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:37.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:37.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:37.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[30018]: Deploying daemon node-exporter.b on smithi193 2024-09-16T08:20:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[23701]: pgmap v23: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:37.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:37 smithi090 ceph-mon[23701]: Deploying daemon node-exporter.b on smithi193 2024-09-16T08:20:37.814 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:37.815 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:38.001 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:37 smithi193 bash[47944]: Getting image source signatures 2024-09-16T08:20:38.002 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:37 smithi193 bash[47944]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-16T08:20:38.002 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:37 smithi193 bash[47944]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-16T08:20:38.002 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:37 smithi193 bash[47944]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-16T08:20:38.173 INFO:teuthology.orchestra.run.smithi090.stdout:244813135887 2024-09-16T08:20:38.174 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T08:20:39.302 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:39.302 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:39.304 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 bash[47944]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-16T08:20:39.304 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 bash[47944]: Writing manifest to image destination 2024-09-16T08:20:39.560 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-mon[25614]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:39.561 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 podman[47944]: 2024-09-16 08:20:39.435731185 +0000 UTC m=+2.302885772 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-16T08:20:39.561 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 podman[47944]: 2024-09-16 08:20:39.542480632 +0000 UTC m=+2.409635219 container create 46a72e78b0a242221f7e1e0acdb959b162853ca71015c25d0e391e3d13ac775c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T08:20:39.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:39 smithi090 ceph-mon[30018]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:39.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:39 smithi090 ceph-mon[23701]: pgmap v24: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:39.895 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:39 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:20:39] "GET /metrics HTTP/1.1" 200 26198 "" "Prometheus/2.43.0" 2024-09-16T08:20:40.252 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 podman[47944]: 2024-09-16 08:20:39.811577883 +0000 UTC m=+2.678732474 container init 46a72e78b0a242221f7e1e0acdb959b162853ca71015c25d0e391e3d13ac775c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T08:20:40.252 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 podman[47944]: 2024-09-16 08:20:39.814429269 +0000 UTC m=+2.681583856 container start 46a72e78b0a242221f7e1e0acdb959b162853ca71015c25d0e391e3d13ac775c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T08:20:40.252 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.818Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.818Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T08:20:40.253 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T08:20:40.254 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T08:20:40.254 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T08:20:40.254 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T08:20:40.254 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T08:20:40.254 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T08:20:40.254 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T08:20:40.254 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T08:20:40.254 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.819Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T08:20:40.255 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T08:20:40.256 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T08:20:40.257 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T08:20:40.257 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T08:20:40.257 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T08:20:40.257 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T08:20:40.257 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-16T08:20:40.257 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b[48019]: ts=2024-09-16T08:20:39.820Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T08:20:40.257 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 bash[47944]: 46a72e78b0a242221f7e1e0acdb959b162853ca71015c25d0e391e3d13ac775c 2024-09-16T08:20:40.257 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:20:39 smithi193 systemd[1]: Started Ceph node-exporter.b for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:20:40.294 INFO:teuthology.orchestra.run.smithi090.stdout:120259084325 2024-09-16T08:20:40.295 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T08:20:41.209 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.209 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.209 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.209 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.209 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.209 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:20:41.209 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:20:41.209 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[30018]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:20:41.210 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:40 smithi090 ceph-mon[23701]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:41.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:41.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:20:41.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:20:41.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:20:41.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:40 smithi193 ceph-mon[25614]: pgmap v25: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:42.251 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:42 smithi193 ceph-mon[25614]: Reconfiguring daemon agent.smithi193 on smithi193 2024-09-16T08:20:42.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:42.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:42.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:42.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:42.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[30018]: Reconfiguring daemon agent.smithi193 on smithi193 2024-09-16T08:20:42.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:42.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:42.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:42.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:42.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[23701]: Reconfiguring daemon agent.smithi193 on smithi193 2024-09-16T08:20:42.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:42.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:42.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:42.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:43.262 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:43 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:43.262 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:43 smithi090 ceph-mon[30018]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:43.263 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:43 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:43.263 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:43 smithi090 ceph-mon[23701]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:43.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:43 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:43.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:43 smithi193 ceph-mon[25614]: pgmap v26: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:44.285 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:44.309 INFO:teuthology.orchestra.run.smithi090.stdout:210453397526 2024-09-16T08:20:44.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T08:20:45.487 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:45.516 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:45 smithi090 ceph-mon[23701]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:45.517 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:45 smithi090 ceph-mon[30018]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:45.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:45 smithi193 ceph-mon[25614]: pgmap v27: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:45.794 INFO:teuthology.orchestra.run.smithi090.stdout:90194313258 2024-09-16T08:20:46.344 INFO:tasks.cephadm.ceph_manager.ceph:need seq 90194313256 got 90194313258 for osd.2 2024-09-16T08:20:46.345 DEBUG:teuthology.parallel:result is None 2024-09-16T08:20:46.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:46 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3956382798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T08:20:46.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:46 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3956382798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T08:20:46.685 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:46 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3956382798' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T08:20:46.686 INFO:teuthology.orchestra.run.smithi090.stdout:176093659163 2024-09-16T08:20:47.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:47 smithi090 ceph-mon[23701]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:47.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:47 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2044129420' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T08:20:47.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:47 smithi090 ceph-mon[30018]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:47.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:47 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2044129420' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T08:20:47.596 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659159 got 176093659163 for osd.5 2024-09-16T08:20:47.597 DEBUG:teuthology.parallel:result is None 2024-09-16T08:20:47.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:47 smithi193 ceph-mon[25614]: pgmap v28: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:47.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:47 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2044129420' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T08:20:48.703 INFO:teuthology.orchestra.run.smithi090.stdout:141733920801 2024-09-16T08:20:48.759 INFO:teuthology.orchestra.run.smithi090.stdout:68719476783 2024-09-16T08:20:49.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:49 smithi193 ceph-mon[25614]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:49.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:49 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2044547776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T08:20:49.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:49 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3094415683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T08:20:49.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 68719476779 got 68719476783 for osd.1 2024-09-16T08:20:49.566 DEBUG:teuthology.parallel:result is None 2024-09-16T08:20:49.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:49 smithi090 ceph-mon[23701]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:49.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:49 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2044547776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T08:20:49.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:49 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3094415683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T08:20:49.657 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:49 smithi090 ceph-mon[30018]: pgmap v29: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:49.657 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:49 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2044547776' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T08:20:49.657 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:49 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3094415683' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T08:20:49.939 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:49 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:20:49] "GET /metrics HTTP/1.1" 200 26201 "" "Prometheus/2.43.0" 2024-09-16T08:20:50.430 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:50.430 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:20:50.804 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920798 got 141733920801 for osd.4 2024-09-16T08:20:50.804 DEBUG:teuthology.parallel:result is None 2024-09-16T08:20:51.269 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:51 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.269 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:51 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.269 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:51 smithi193 ceph-mon[25614]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:51.269 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:51 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.269 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:51 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.425 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.425 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.425 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[30018]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:51.425 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.425 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[23701]: pgmap v30: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:51.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:51.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:51 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:53.598 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:53 smithi090 ceph-mon[30018]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:53.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:53 smithi090 ceph-mon[23701]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:53.642 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:53 smithi193 ceph-mon[25614]: pgmap v31: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:54.060 INFO:teuthology.orchestra.run.smithi090.stdout:55834574897 2024-09-16T08:20:54.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2140467218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T08:20:54.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:54.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:54.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:20:54.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:20:54.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:54.616 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2140467218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T08:20:54.616 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:54.616 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:54.616 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:20:54.616 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:20:54.616 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:54.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574892 got 55834574897 for osd.0 2024-09-16T08:20:54.640 DEBUG:teuthology.parallel:result is None 2024-09-16T08:20:54.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:54 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2140467218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T08:20:54.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:54.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:54.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:20:54.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:20:54.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:55.632 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi193 2024-09-16T08:20:55.632 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[30018]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[30018]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[30018]: Reconfiguring daemon alertmanager.a on smithi090 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[30018]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi193 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[23701]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[23701]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[23701]: Reconfiguring daemon alertmanager.a on smithi090 2024-09-16T08:20:55.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:55 smithi090 ceph-mon[23701]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-16T08:20:55.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:55 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi193 2024-09-16T08:20:55.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:55 smithi193 ceph-mon[25614]: pgmap v32: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:55.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:55 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:20:55.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:55 smithi193 ceph-mon[25614]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T08:20:55.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:55 smithi193 ceph-mon[25614]: Reconfiguring daemon alertmanager.a on smithi090 2024-09-16T08:20:55.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:55 smithi193 ceph-mon[25614]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-16T08:20:56.875 INFO:teuthology.orchestra.run.smithi090.stdout:244813135891 2024-09-16T08:20:57.078 INFO:teuthology.orchestra.run.smithi090.stdout:210453397529 2024-09-16T08:20:57.560 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135887 got 244813135891 for osd.7 2024-09-16T08:20:57.560 DEBUG:teuthology.parallel:result is None 2024-09-16T08:20:57.644 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[30018]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:57.644 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:57.644 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:57.644 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/140757337' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T08:20:57.644 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1863946332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T08:20:57.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[23701]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:57.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:57.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:57.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/140757337' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T08:20:57.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:57 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1863946332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T08:20:57.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:57 smithi193 ceph-mon[25614]: pgmap v33: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:57.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:20:57.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:20:57.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:57 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/140757337' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T08:20:57.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:57 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1863946332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T08:20:58.364 INFO:teuthology.orchestra.run.smithi090.stdout:120259084329 2024-09-16T08:20:58.897 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397526 got 210453397529 for osd.6 2024-09-16T08:20:58.897 DEBUG:teuthology.parallel:result is None 2024-09-16T08:20:59.521 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:59 smithi090 ceph-mon[30018]: pgmap v34: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:59.521 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:20:59 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1567898995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T08:20:59.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:59 smithi090 ceph-mon[23701]: pgmap v34: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:59.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:20:59 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1567898995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T08:20:59.545 INFO:tasks.cephadm.ceph_manager.ceph:need seq 120259084325 got 120259084329 for osd.3 2024-09-16T08:20:59.546 DEBUG:teuthology.parallel:result is None 2024-09-16T08:20:59.546 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T08:20:59.546 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T08:20:59.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:59 smithi193 ceph-mon[25614]: pgmap v34: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:20:59.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:20:59 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1567898995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T08:20:59.855 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:00.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:20:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:20:59] "GET /metrics HTTP/1.1" 200 26273 "" "Prometheus/2.43.0" 2024-09-16T08:21:01.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:01 smithi090 ceph-mon[23701]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:01.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:01.709 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:01 smithi090 ceph-mon[30018]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:01.709 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:01.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:01 smithi193 ceph-mon[25614]: pgmap v35: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:01.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:03.551 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:03 smithi090 ceph-mon[30018]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:03.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:03 smithi090 ceph-mon[23701]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:03.629 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:03 smithi193 ceph-mon[25614]: pgmap v36: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:04.812 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:04 smithi090 systemd[1]: Stopping Ceph alertmanager.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:21:05.414 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:05 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[51544]: ts=2024-09-16T08:21:05.136Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T08:21:05.414 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:05 smithi090 podman[56027]: 2024-09-16 08:21:05.156875611 +0000 UTC m=+0.135268221 container died 12a52cca346913162bdb9ccb03a4e6b5fbb0f6177b338dd0375a7cf56b25c2ec (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:05.614 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:05 smithi193 ceph-mon[25614]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:05.680 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:05 smithi090 ceph-mon[30018]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:05.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:05 smithi090 ceph-mon[23701]: pgmap v37: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:05.681 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:05 smithi090 podman[56027]: 2024-09-16 08:21:05.521640353 +0000 UTC m=+0.500032962 container cleanup 12a52cca346913162bdb9ccb03a4e6b5fbb0f6177b338dd0375a7cf56b25c2ec (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:05.681 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:05 smithi090 bash[56027]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T08:21:06.312 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:05 smithi090 podman[56080]: 2024-09-16 08:21:05.975575217 +0000 UTC m=+0.812762399 container remove 12a52cca346913162bdb9ccb03a4e6b5fbb0f6177b338dd0375a7cf56b25c2ec (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:06.312 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:06 smithi090 podman[56080]: 2024-09-16 08:21:06.067916802 +0000 UTC m=+0.905103984 volume remove 1515c2d64d556a0dc33d38887534436b07d6fcf482a61c1b94b72df408cf36dd 2024-09-16T08:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:21:06.521 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-09-16T08:21:06.562 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:06 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T08:21:06.562 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:06 smithi090 systemd[1]: Stopped Ceph alertmanager.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:21:06.562 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:06 smithi090 systemd[1]: Starting Ceph alertmanager.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:21:07.070 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-16T08:21:05.680633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":223020,"kb_used_data":6484,"kb_used_omap":15,"kb_used_meta":216432,"kb_avail":749738196,"statfs":{"total":767960285184,"available":767731912704,"internally_reserved":0,"allocated":6639616,"data_stored":3735930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15554,"internal_metadata":221627198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002448"},"pg_stats":[{"pgid":"1.0","version":"62'128","reported_seq":255,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T08:19:58.336117+0000","last_change":"2024-09-16T08:19:29.098582+0000","last_active":"2024-09-16T08:19:58.336117+0000","last_peered":"2024-09-16T08:19:58.336117+0000","last_clean":"2024-09-16T08:19:58.336117+0000","last_became_active":"2024-09-16T08:19:29.098194+0000","last_became_peered":"2024-09-16T08:19:29.098194+0000","last_unstale":"2024-09-16T08:19:58.336117+0000","last_undegraded":"2024-09-16T08:19:58.336117+0000","last_fullsized":"2024-09-16T08:19:58.336117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:17:26.343501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:17:26.343501+0000","last_clean_scrub_stamp":"2024-09-16T08:17:26.343501+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:35.779464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1200128,"data_stored":835903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.621},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.5}]},{"osd":2,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.69499999999999995}]},{"osd":5,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.59799999999999998}]}]},{"osd":6,"up_from":49,"seq":210453397531,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1200128,"data_stored":835903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53800000000000003}]},{"osd":1,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.45700000000000002}]},{"osd":2,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.498}]},{"osd":3,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.51000000000000001}]},{"osd":4,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.40799999999999997}]},{"osd":5,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.441},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.55900000000000005}]},{"osd":7,"last update":"Mon Sep 16 08:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.47999999999999998}]}]},{"osd":5,"up_from":41,"seq":176093659167,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717540,"statfs":{"total":95995035648,"available":95966760960,"internally_reserved":0,"allocated":606208,"data_stored":245535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69099999999999995,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.69899999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64100000000000001}]},{"osd":1,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59999999999999998}]},{"osd":2,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.439,"5min":0.374,"15min":0.374},"max":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.622}]},{"osd":3,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.749,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68100000000000005}]},{"osd":4,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41799999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.68799999999999994,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.35699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47599999999999998}]},{"osd":6,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46300000000000002}]},{"osd":7,"last update":"Mon Sep 16 08:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.496}]}]},{"osd":4,"up_from":33,"seq":141733920804,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717536,"statfs":{"total":95995035648,"available":95966756864,"internally_reserved":0,"allocated":610304,"data_stored":245808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.64200000000000002}]},{"osd":1,"last update":"Mon Sep 16 08:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.63400000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.504,"15min":0.5},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.66000000000000003}]},{"osd":2,"last update":"Mon Sep 16 08:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.69299999999999995}]},{"osd":3,"last update":"Mon Sep 16 08:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.45600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.38100000000000001},"min":{"1min":0.29899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36599999999999999}]},{"osd":6,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.45000000000000001}]},{"osd":7,"last update":"Mon Sep 16 08:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.42399999999999999}]}]},{"osd":3,"up_from":28,"seq":120259084330,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717540,"statfs":{"total":95995035648,"available":95966760960,"internally_reserved":0,"allocated":606208,"data_stored":245535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.36599999999999999,"15min":0.375},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.47499999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.315},{"interface":"front","average":{"1min":0.37,"5min":0.372,"15min":0.373},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.55800000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.45700000000000002}]},{"osd":1,"last update":"Mon Sep 16 08:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.499,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.374,"15min":0.376},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.36099999999999999}]},{"osd":2,"last update":"Mon Sep 16 08:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38300000000000001,"15min":0.38600000000000001},"min":{"1min":0.26900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.379}]},{"osd":4,"last update":"Mon Sep 16 08:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Sep 16 08:20:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.89900000000000002,"15min":0.95999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80500000000000005,"5min":8.8759999999999994,"15min":8.8759999999999994},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63800000000000001}]},{"osd":6,"last update":"Mon Sep 16 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.79600000000000004}]},{"osd":7,"last update":"Mon Sep 16 08:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":8.9000000000000004,"5min":8.9000000000000004,"15min":8.9000000000000004},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80800000000000005}]}]},{"osd":2,"up_from":21,"seq":90194313262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717536,"statfs":{"total":95995035648,"available":95966756864,"internally_reserved":0,"allocated":610304,"data_stored":245808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.371,"15min":0.372},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.51700000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.307},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.215,"15min":0.215},"max":{"1min":0.59399999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.317}]},{"osd":1,"last update":"Mon Sep 16 08:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.30199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55900000000000005,"5min":0.625,"15min":0.625},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.39200000000000002,"15min":0.39800000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55000000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.26700000000000002}]},{"osd":3,"last update":"Mon Sep 16 08:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.436,"15min":0.44500000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54400000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.42299999999999999,"15min":0.42999999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57099999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.35299999999999998}]},{"osd":4,"last update":"Mon Sep 16 08:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.745,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Sep 16 08:20:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":1.595,"5min":0.747,"15min":0.60599999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":19.356999999999999,"5min":19.356999999999999,"15min":19.356999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Mon Sep 16 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.80100000000000005}]},{"osd":7,"last update":"Mon Sep 16 08:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.64300000000000002}]}]},{"osd":1,"up_from":16,"seq":68719476786,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717540,"statfs":{"total":95995035648,"available":95966760960,"internally_reserved":0,"allocated":606208,"data_stored":245535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.314},"min":{"1min":0.20599999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.221},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.35599999999999998,"15min":0.32700000000000001},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.34499999999999997}]},{"osd":2,"last update":"Mon Sep 16 08:20:31 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.374,"15min":0.37},"min":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.29399999999999998}]},{"osd":3,"last update":"Mon Sep 16 08:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.437,"15min":0.44400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.314},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.32200000000000001}]},{"osd":4,"last update":"Mon Sep 16 08:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.748,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.36199999999999999}]},{"osd":5,"last update":"Mon Sep 16 08:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.50700000000000001}]},{"osd":6,"last update":"Mon Sep 16 08:20:02 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.376},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.433}]},{"osd":7,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.38600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1172,"kb_used_omap":4,"kb_used_meta":27387,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1200128,"data_stored":835903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4404,"internal_metadata":28045004},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 08:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33300000000000002,"15min":0.32400000000000001},"min":{"1min":0.218,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.49099999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.34499999999999997}]},{"osd":2,"last update":"Mon Sep 16 08:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.376,"15min":0.376},"min":{"1min":0.245,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.49099999999999999,"5min":0.622,"15min":0.622},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.433,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.30499999999999999}]},{"osd":3,"last update":"Mon Sep 16 08:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46700000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.29899999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.32400000000000001}]},{"osd":4,"last update":"Mon Sep 16 08:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67000000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.39800000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55800000000000005}]},{"osd":5,"last update":"Mon Sep 16 08:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72199999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.74199999999999999,"5min":0.747,"15min":0.747},"last":0.58799999999999997}]},{"osd":6,"last update":"Mon Sep 16 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.38}]},{"osd":7,"last update":"Mon Sep 16 08:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.40899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T08:21:07.072 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T08:21:07.311 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:07 smithi090 podman[56318]: 2024-09-16 08:21:07.115543149 +0000 UTC m=+0.547770511 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-16T08:21:07.586 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:07 smithi090 ceph-mon[30018]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:07.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:07 smithi090 ceph-mon[23701]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:07.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:07 smithi193 ceph-mon[25614]: pgmap v38: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:07.878 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:07 smithi090 podman[56318]: 2024-09-16 08:21:07.587144914 +0000 UTC m=+1.019372272 volume create 76af2c5267aafb6b458ae6cfd1d82bc018a5f1da70c790f5ccbc914f9a029dda 2024-09-16T08:21:07.878 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:07 smithi090 podman[56318]: 2024-09-16 08:21:07.679124226 +0000 UTC m=+1.111351581 container create 3c66ea902fdc3b2c45526b3c322e0b93abcb5b1cd3546d9c228b59303eaf0137 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:08.281 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:07 smithi090 podman[56318]: 2024-09-16 08:21:07.975442595 +0000 UTC m=+1.407669959 container init 3c66ea902fdc3b2c45526b3c322e0b93abcb5b1cd3546d9c228b59303eaf0137 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:08.281 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:07 smithi090 podman[56318]: 2024-09-16 08:21:07.979132498 +0000 UTC m=+1.411359853 container start 3c66ea902fdc3b2c45526b3c322e0b93abcb5b1cd3546d9c228b59303eaf0137 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:08.281 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:07 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:07.991Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-16T08:21:08.281 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:07 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:07.991Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-16T08:21:08.281 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:07 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:07.992Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.90 port=9094 2024-09-16T08:21:08.281 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:07 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:07.993Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T08:21:08.281 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:08.024Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T08:21:08.282 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:08.024Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T08:21:08.282 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:08.027Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-16T08:21:08.282 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:08.027Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T08:21:08.282 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:08 smithi090 bash[56318]: 3c66ea902fdc3b2c45526b3c322e0b93abcb5b1cd3546d9c228b59303eaf0137 2024-09-16T08:21:08.282 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:08 smithi090 systemd[1]: Started Ceph alertmanager.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:21:08.520 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:08.550 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-mon[30018]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:21:08.550 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:08.550 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:08.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-mon[23701]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:21:08.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:08.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:08.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:08 smithi193 ceph-mon[25614]: from='client.14619 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:21:08.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:08.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:09.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:09 smithi193 ceph-mon[25614]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:09.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:09 smithi193 ceph-mon[25614]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:21:09.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:09 smithi193 ceph-mon[25614]: Reconfiguring daemon prometheus.a on smithi193 2024-09-16T08:21:09.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:09 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:09.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-mon[30018]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:09.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-mon[30018]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:21:09.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-mon[30018]: Reconfiguring daemon prometheus.a on smithi193 2024-09-16T08:21:09.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:09.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-mon[23701]: pgmap v39: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:09.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-mon[23701]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:21:09.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-mon[23701]: Reconfiguring daemon prometheus.a on smithi193 2024-09-16T08:21:09.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:10.042 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:21:09] "GET /metrics HTTP/1.1" 200 26273 "" "Prometheus/2.43.0" 2024-09-16T08:21:10.042 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:09.994Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000148276s 2024-09-16T08:21:10.906 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:21:10.907 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-09-16T08:21:11.472 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 systemd[1]: Stopping Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:21:11.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-mon[25614]: pgmap v40: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:11.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:11.752 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.641Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T08:21:11.752 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-16T08:21:11.752 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-16T08:21:11.752 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T08:21:11.752 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-16T08:21:11.753 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-16T08:21:11.753 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T08:21:11.753 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-16T08:21:11.753 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-16T08:21:11.753 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T08:21:11.753 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-16T08:21:11.753 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[44840]: ts=2024-09-16T08:21:11.642Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-16T08:21:11.753 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:11 smithi193 podman[52571]: 2024-09-16 08:21:11.652260659 +0000 UTC m=+0.145002180 container died 9b7edc7a6ae843e78ac9085d231f2cfb1b39e9b029a6a1a6dd10ab370bfdac1a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:11.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:11 smithi090 ceph-mon[30018]: pgmap v40: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:11.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:11 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:11.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:11 smithi090 ceph-mon[23701]: pgmap v40: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:11.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:11 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:12.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 podman[52571]: 2024-09-16 08:21:12.016331427 +0000 UTC m=+0.509072951 container cleanup 9b7edc7a6ae843e78ac9085d231f2cfb1b39e9b029a6a1a6dd10ab370bfdac1a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:12.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 bash[52571]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T08:21:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:12 smithi090 ceph-mon[23701]: from='client.24518 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:21:12.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:12 smithi090 ceph-mon[30018]: from='client.24518 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:21:12.593 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-mon[25614]: from='client.24518 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:21:12.593 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 podman[52582]: 2024-09-16 08:21:12.259391147 +0000 UTC m=+0.609569512 container remove 9b7edc7a6ae843e78ac9085d231f2cfb1b39e9b029a6a1a6dd10ab370bfdac1a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:12.593 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-16T08:21:12.593 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 systemd[1]: Stopped Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:21:12.593 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.173s CPU time. 2024-09-16T08:21:12.593 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 systemd[1]: Starting Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:21:12.853 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 podman[52677]: 2024-09-16 08:21:12.496583772 +0000 UTC m=+0.016821623 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-16T08:21:12.854 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 podman[52677]: 2024-09-16 08:21:12.627001027 +0000 UTC m=+0.147238867 container create 0416160d55ca4337bb9e5160aae3d3731563c9caf5deed41b77dd35208f01173 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:12.854 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 podman[52677]: 2024-09-16 08:21:12.854149222 +0000 UTC m=+0.374387065 container init 0416160d55ca4337bb9e5160aae3d3731563c9caf5deed41b77dd35208f01173 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:13.248 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:12 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:12] ENGINE Bus STOPPING 2024-09-16T08:21:13.249 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:21:13.249 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STOPPED 2024-09-16T08:21:13.249 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STARTING 2024-09-16T08:21:13.249 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Serving on http://:::9283 2024-09-16T08:21:13.249 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STARTED 2024-09-16T08:21:13.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 podman[52677]: 2024-09-16 08:21:12.856926784 +0000 UTC m=+0.377164626 container start 0416160d55ca4337bb9e5160aae3d3731563c9caf5deed41b77dd35208f01173 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:21:13.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.881Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-16T08:21:13.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.881Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-16T08:21:13.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.881Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi193 (none))" 2024-09-16T08:21:13.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.881Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.881Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.883Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.883Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.884Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.884Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.886Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.886Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.476µs 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.886Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.887Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.888Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-16T08:21:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.888Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=18.56µs wal_replay_duration=1.503177ms wbl_replay_duration=92ns total_replay_duration=1.556065ms 2024-09-16T08:21:13.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.888Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T08:21:13.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.888Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-16T08:21:13.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.888Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T08:21:13.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.907Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=18.982025ms db_storage=726ns remote_storage=964ns web_handler=353ns query_engine=597ns scrape=390.726µs scrape_sd=78.526µs notify=9.379µs notify_sd=10.964µs rules=18.182182ms tracing=6.501µs 2024-09-16T08:21:13.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.907Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-16T08:21:13.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:21:12.907Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T08:21:13.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 bash[52677]: 0416160d55ca4337bb9e5160aae3d3731563c9caf5deed41b77dd35208f01173 2024-09-16T08:21:13.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:21:12 smithi193 systemd[1]: Started Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:21:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: pgmap v41: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:21:13.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.564 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STOPPING 2024-09-16T08:21:13.564 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:21:13.564 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STOPPED 2024-09-16T08:21:13.564 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STARTING 2024-09-16T08:21:13.564 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Serving on http://:::9283 2024-09-16T08:21:13.564 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STARTED 2024-09-16T08:21:13.564 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STOPPING 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: pgmap v41: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:13.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:13.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:21:13.566 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: pgmap v41: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:21:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:21:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:21:13.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:13 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:13.898 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-16T08:21:09.681430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":223020,"kb_used_data":6484,"kb_used_omap":15,"kb_used_meta":216432,"kb_avail":749738196,"statfs":{"total":767960285184,"available":767731912704,"internally_reserved":0,"allocated":6639616,"data_stored":3735930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15554,"internal_metadata":221627198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002509"},"pg_stats":[{"pgid":"1.0","version":"62'128","reported_seq":255,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-16T08:19:58.336117+0000","last_change":"2024-09-16T08:19:29.098582+0000","last_active":"2024-09-16T08:19:58.336117+0000","last_peered":"2024-09-16T08:19:58.336117+0000","last_clean":"2024-09-16T08:19:58.336117+0000","last_became_active":"2024-09-16T08:19:29.098194+0000","last_became_peered":"2024-09-16T08:19:29.098194+0000","last_unstale":"2024-09-16T08:19:58.336117+0000","last_undegraded":"2024-09-16T08:19:58.336117+0000","last_fullsized":"2024-09-16T08:19:58.336117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:17:26.343501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:17:26.343501+0000","last_clean_scrub_stamp":"2024-09-16T08:17:26.343501+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:26:35.779464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1200128,"data_stored":835903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.64000000000000001}]},{"osd":1,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.621},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.5}]},{"osd":2,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67500000000000004}]},{"osd":3,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.69499999999999995}]},{"osd":5,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.59799999999999998}]}]},{"osd":6,"up_from":49,"seq":210453397532,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1200128,"data_stored":835903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.48899999999999999}]},{"osd":1,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.50600000000000001}]},{"osd":2,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.498}]},{"osd":3,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.44800000000000001}]},{"osd":4,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.35599999999999998}]},{"osd":5,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.40600000000000003}]},{"osd":7,"last update":"Mon Sep 16 08:20:27 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.47999999999999998}]}]},{"osd":5,"up_from":41,"seq":176093659168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717540,"statfs":{"total":95995035648,"available":95966760960,"internally_reserved":0,"allocated":606208,"data_stored":245535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69099999999999995,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.42299999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.69899999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55300000000000005}]},{"osd":1,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.7959999999999998,"5min":2.7959999999999998,"15min":2.7959999999999998},"last":0.5},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.439,"5min":0.374,"15min":0.374},"max":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.625},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.54200000000000004,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.54700000000000004}]},{"osd":3,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.749,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58199999999999996}]},{"osd":4,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.41799999999999998,"15min":0.41599999999999998},"min":{"1min":0.29599999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.68799999999999994,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.35699999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.47799999999999998}]},{"osd":6,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56599999999999995}]},{"osd":7,"last update":"Mon Sep 16 08:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.50800000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920805,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717536,"statfs":{"total":95995035648,"available":95966756864,"internally_reserved":0,"allocated":610304,"data_stored":245808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.5,"15min":0.498},"min":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.50900000000000001,"15min":0.505},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.59099999999999997}]},{"osd":1,"last update":"Mon Sep 16 08:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.63400000000000001,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.504,"15min":0.5},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.49099999999999999}]},{"osd":2,"last update":"Mon Sep 16 08:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.39800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.52900000000000003}]},{"osd":3,"last update":"Mon Sep 16 08:20:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.45600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Mon Sep 16 08:20:40 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.38400000000000001,"15min":0.38100000000000001},"min":{"1min":0.29899999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Mon Sep 16 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.623},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.57799999999999996}]},{"osd":7,"last update":"Mon Sep 16 08:20:25 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.54400000000000004}]}]},{"osd":3,"up_from":28,"seq":120259084331,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717540,"statfs":{"total":95995035648,"available":95966760960,"internally_reserved":0,"allocated":606208,"data_stored":245535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.36599999999999999,"15min":0.375},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.47499999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.373},{"interface":"front","average":{"1min":0.37,"5min":0.372,"15min":0.373},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.55800000000000005,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.38400000000000001}]},{"osd":1,"last update":"Mon Sep 16 08:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.499,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.373,"5min":0.374,"15min":0.376},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.35399999999999998}]},{"osd":2,"last update":"Mon Sep 16 08:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.40999999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.38300000000000001,"15min":0.38600000000000001},"min":{"1min":0.26900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.39500000000000002}]},{"osd":4,"last update":"Mon Sep 16 08:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.52500000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.504},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.36299999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.495}]},{"osd":5,"last update":"Mon Sep 16 08:20:39 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.89900000000000002,"15min":0.95999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80500000000000005,"5min":8.8759999999999994,"15min":8.8759999999999994},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.53100000000000003,"15min":0.52300000000000002},"min":{"1min":0.40400000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.46899999999999997}]},{"osd":6,"last update":"Mon Sep 16 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41599999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.80200000000000005,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54000000000000004}]},{"osd":7,"last update":"Mon Sep 16 08:20:28 2024","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":8.9000000000000004,"5min":8.9000000000000004,"15min":8.9000000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.53300000000000003}]}]},{"osd":2,"up_from":21,"seq":90194313263,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27616,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717536,"statfs":{"total":95995035648,"available":95966756864,"internally_reserved":0,"allocated":610304,"data_stored":245808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.371,"15min":0.372},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.51700000000000002,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.372,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.215,"15min":0.215},"max":{"1min":0.59399999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.29899999999999999}]},{"osd":1,"last update":"Mon Sep 16 08:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39500000000000002,"15min":0.38900000000000001},"min":{"1min":0.30199999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55900000000000005,"5min":0.625,"15min":0.625},"last":0.314},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.39200000000000002,"15min":0.39800000000000002},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.55000000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.35599999999999998}]},{"osd":3,"last update":"Mon Sep 16 08:20:54 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.436,"15min":0.44500000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.54400000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.42299999999999999,"15min":0.42999999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57099999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.33700000000000002}]},{"osd":4,"last update":"Mon Sep 16 08:20:14 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55400000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.745,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65800000000000003}]},{"osd":5,"last update":"Mon Sep 16 08:20:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":1.595,"5min":0.747,"15min":0.60599999999999998},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":19.356999999999999,"5min":19.356999999999999,"15min":19.356999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Mon Sep 16 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57499999999999996}]},{"osd":7,"last update":"Mon Sep 16 08:20:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56699999999999995}]}]},{"osd":1,"up_from":16,"seq":68719476787,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27612,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717540,"statfs":{"total":95995035648,"available":95966760960,"internally_reserved":0,"allocated":606208,"data_stored":245535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.314},"min":{"1min":0.20599999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.35599999999999998,"15min":0.32700000000000001},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.38600000000000001}]},{"osd":2,"last update":"Mon Sep 16 08:20:31 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.374,"15min":0.37},"min":{"1min":0.252,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.38100000000000001},"min":{"1min":0.25,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.34100000000000003}]},{"osd":3,"last update":"Mon Sep 16 08:20:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.437,"15min":0.44400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.41099999999999998}]},{"osd":4,"last update":"Mon Sep 16 08:20:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57299999999999995,"15min":0.57499999999999996},"min":{"1min":0.38200000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.748,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72599999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Mon Sep 16 08:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56799999999999995,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.751},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.375,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.64300000000000002}]},{"osd":6,"last update":"Mon Sep 16 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81999999999999995,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.621},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.61199999999999999,"15min":0.61399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80000000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.72299999999999998}]},{"osd":7,"last update":"Mon Sep 16 08:20:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28576,"kb_used_data":1172,"kb_used_omap":4,"kb_used_meta":27387,"kb_avail":93716576,"statfs":{"total":95995035648,"available":95965773824,"internally_reserved":0,"allocated":1200128,"data_stored":835903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4404,"internal_metadata":28045004},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 08:20:09 2024","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.33300000000000002,"15min":0.32400000000000001},"min":{"1min":0.218,"5min":0.16800000000000001,"15min":0.16800000000000001},"max":{"1min":0.49099999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.36299999999999999,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.218}]},{"osd":2,"last update":"Mon Sep 16 08:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.376,"15min":0.376},"min":{"1min":0.245,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.49099999999999999,"5min":0.622,"15min":0.622},"last":0.246},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.36799999999999999,"15min":0.372},"min":{"1min":0.23999999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.433,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.28599999999999998}]},{"osd":3,"last update":"Mon Sep 16 08:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.247,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.46700000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.42299999999999999,"15min":0.43099999999999999},"min":{"1min":0.29899999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.46600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.27400000000000002}]},{"osd":4,"last update":"Mon Sep 16 08:20:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67000000000000004,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.39800000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.47899999999999998}]},{"osd":5,"last update":"Mon Sep 16 08:20:38 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.53000000000000003,"15min":0.53400000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72199999999999998,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.74199999999999999,"5min":0.747,"15min":0.747},"last":0.502}]},{"osd":6,"last update":"Mon Sep 16 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.56299999999999994}]},{"osd":7,"last update":"Mon Sep 16 08:20:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.51100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T08:21:13.900 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T08:21:13.900 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T08:21:13.900 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T08:21:13.901 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T08:21:14.113 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:14.229 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:21:14.229 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STOPPED 2024-09-16T08:21:14.230 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STARTING 2024-09-16T08:21:14.230 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Serving on http://:::9283 2024-09-16T08:21:14.230 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:13 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:21:13] ENGINE Bus STARTED 2024-09-16T08:21:14.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:21:14.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:21:14.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:21:14.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:21:14.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:21:14.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:21:14.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: pgmap v42: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:14.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:21:14.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-16T08:21:14.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:14 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:21:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:21:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:21:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:21:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:21:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:21:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:21:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: pgmap v42: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi193.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi193.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: pgmap v42: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:14.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:21:14.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-16T08:21:14.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:14 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:21:15.885 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-16T08:21:15.886 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T08:21:16.739 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:16 smithi090 ceph-mon[30018]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:16.739 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:16 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3451448531' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T08:21:16.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:16 smithi090 ceph-mon[23701]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:16.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:16 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3451448531' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T08:21:16.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:16 smithi193 ceph-mon[25614]: pgmap v43: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:16.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:16 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3451448531' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T08:21:17.138 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T08:21:17.139 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T08:21:17.139 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T08:21:17.149 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-09-16T08:21:17.149 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'radosgw-admin realm create --rgw-realm=r --default' 2024-09-16T08:21:17.884 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:18.030 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:21:17 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:21:17.995Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001442403s 2024-09-16T08:21:18.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:18 smithi193 ceph-mon[25614]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:21:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[23701]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:18.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:18.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:18.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:21:18.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[30018]: pgmap v44: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:18.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:18.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:18.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:18.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:21:18.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:20.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:19 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:21:19] "GET /metrics HTTP/1.1" 200 26269 "" "Prometheus/2.43.0" 2024-09-16T08:21:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:20 smithi193 ceph-mon[25614]: pgmap v45: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:20 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:20 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:20 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:20.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:20 smithi090 ceph-mon[23701]: pgmap v45: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:20.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:20 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:20.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:20 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:20.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:20 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:20.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:20 smithi090 ceph-mon[30018]: pgmap v45: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:20.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:20 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:20.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:20 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:20.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:20 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:21.541 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:21:21.542 INFO:teuthology.orchestra.run.smithi090.stdout: "id": "c8d396c0-97e8-49c7-a830-c8c33caf2528", 2024-09-16T08:21:21.542 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "r", 2024-09-16T08:21:21.542 INFO:teuthology.orchestra.run.smithi090.stdout: "current_period": "fe82523e-e900-4a4e-a386-ac76437cbd2d", 2024-09-16T08:21:21.542 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 1 2024-09-16T08:21:21.542 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:21:21.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:21 smithi090 ceph-mon[30018]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T08:21:21.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:21 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2236935801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T08:21:21.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:21 smithi090 ceph-mon[30018]: pgmap v47: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:21.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:21 smithi090 ceph-mon[23701]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T08:21:21.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:21 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2236935801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T08:21:21.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:21 smithi090 ceph-mon[23701]: pgmap v47: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:21.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:21 smithi193 ceph-mon[25614]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T08:21:21.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:21 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2236935801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-16T08:21:21.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:21 smithi193 ceph-mon[25614]: pgmap v47: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:22.390 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'radosgw-admin zonegroup create --rgw-zonegroup=default --master --default' 2024-09-16T08:21:22.623 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:22 smithi090 ceph-mon[30018]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:21:22.623 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:22 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2236935801' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T08:21:22.623 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:22 smithi090 ceph-mon[30018]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T08:21:22.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:22 smithi090 ceph-mon[23701]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:21:22.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:22 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2236935801' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T08:21:22.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:22 smithi090 ceph-mon[23701]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T08:21:22.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:22 smithi193 ceph-mon[25614]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:21:22.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:22 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2236935801' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T08:21:22.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:22 smithi193 ceph-mon[25614]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T08:21:23.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:23 smithi090 ceph-mon[30018]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T08:21:23.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:23 smithi090 ceph-mon[30018]: pgmap v50: 33 pgs: 33 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-16T08:21:23.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:23 smithi090 ceph-mon[23701]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T08:21:23.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:23 smithi090 ceph-mon[23701]: pgmap v50: 33 pgs: 33 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-16T08:21:23.827 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:23 smithi193 ceph-mon[25614]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T08:21:23.827 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:23 smithi193 ceph-mon[25614]: pgmap v50: 33 pgs: 33 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-16T08:21:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:24.656 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:24 smithi193 ceph-mon[25614]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:21:24.656 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:24 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:21:24.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:24 smithi090 ceph-mon[30018]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:21:24.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:24 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:21:24.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:24 smithi090 ceph-mon[23701]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:21:24.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:24 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:21:25.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:25 smithi090 ceph-mon[23701]: pgmap v51: 33 pgs: 33 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-16T08:21:25.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:25 smithi090 ceph-mon[30018]: pgmap v51: 33 pgs: 33 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-16T08:21:25.980 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:25 smithi193 ceph-mon[25614]: pgmap v51: 33 pgs: 33 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 341 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-16T08:21:26.879 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:21:26.879 INFO:teuthology.orchestra.run.smithi090.stdout: "id": "e7623a06-4b41-4bd9-9c85-b20a608c5a6b", 2024-09-16T08:21:26.879 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default", 2024-09-16T08:21:26.879 INFO:teuthology.orchestra.run.smithi090.stdout: "api_name": "default", 2024-09-16T08:21:26.879 INFO:teuthology.orchestra.run.smithi090.stdout: "is_master": true, 2024-09-16T08:21:26.879 INFO:teuthology.orchestra.run.smithi090.stdout: "endpoints": [], 2024-09-16T08:21:26.879 INFO:teuthology.orchestra.run.smithi090.stdout: "hostnames": [], 2024-09-16T08:21:26.879 INFO:teuthology.orchestra.run.smithi090.stdout: "hostnames_s3website": [], 2024-09-16T08:21:26.879 INFO:teuthology.orchestra.run.smithi090.stdout: "master_zone": "", 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: "zones": [], 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: "placement_targets": [ 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default-placement", 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_classes": [] 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: "default_placement": "default-placement", 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: "realm_id": "c8d396c0-97e8-49c7-a830-c8c33caf2528", 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: "sync_policy": { 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: "groups": [] 2024-09-16T08:21:26.880 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:26.881 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled_features": [ 2024-09-16T08:21:26.881 INFO:teuthology.orchestra.run.smithi090.stdout: "resharding" 2024-09-16T08:21:26.881 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-16T08:21:26.881 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:21:27.502 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:27 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:27.502 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:27 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:27.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:27 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:27.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:27 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:27.527 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default' 2024-09-16T08:21:27.653 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:27 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:27.653 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:27 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:27.729 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:28.468 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:28 smithi090 ceph-mon[30018]: pgmap v52: 33 pgs: 33 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 313 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:21:28.469 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:28 smithi090 ceph-mon[23701]: pgmap v52: 33 pgs: 33 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 313 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:21:28.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:28 smithi193 ceph-mon[25614]: pgmap v52: 33 pgs: 33 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 313 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-16T08:21:29.275 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "id": "c413f657-35f9-42f4-b76d-6912a5217473", 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "z", 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "domain_root": "z.rgw.meta:root", 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "control_pool": "z.rgw.control", 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "gc_pool": "z.rgw.log:gc", 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "lc_pool": "z.rgw.log:lc", 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "log_pool": "z.rgw.log", 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "intent_log_pool": "z.rgw.log:intent", 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "usage_log_pool": "z.rgw.log:usage", 2024-09-16T08:21:29.276 INFO:teuthology.orchestra.run.smithi090.stdout: "roles_pool": "z.rgw.meta:roles", 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "reshard_pool": "z.rgw.log:reshard", 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "user_keys_pool": "z.rgw.meta:users.keys", 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "user_email_pool": "z.rgw.meta:users.email", 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "user_swift_pool": "z.rgw.meta:users.swift", 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "user_uid_pool": "z.rgw.meta:users.uid", 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "otp_pool": "z.rgw.otp", 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "system_key": { 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "access_key": "", 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "secret_key": "" 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: "placement_pools": [ 2024-09-16T08:21:29.277 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: "key": "default-placement", 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: "val": { 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: "index_pool": "z.rgw.buckets.index", 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_classes": { 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: "STANDARD": { 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: "data_pool": "z.rgw.buckets.data" 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: "data_extra_pool": "z.rgw.buckets.non-ec", 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: "index_type": 0, 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: "inline_data": true 2024-09-16T08:21:29.278 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:21:29.279 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:21:29.279 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-16T08:21:29.279 INFO:teuthology.orchestra.run.smithi090.stdout: "realm_id": "c8d396c0-97e8-49c7-a830-c8c33caf2528", 2024-09-16T08:21:29.279 INFO:teuthology.orchestra.run.smithi090.stdout: "notif_pool": "z.rgw.log:notif" 2024-09-16T08:21:29.279 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:21:29.888 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:29 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:21:29] "GET /metrics HTTP/1.1" 200 28590 "" "Prometheus/2.43.0" 2024-09-16T08:21:29.909 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'radosgw-admin period update --rgw-realm=r --commit' 2024-09-16T08:21:30.106 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:30.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:30 smithi090 ceph-mon[23701]: pgmap v53: 33 pgs: 33 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:21:30.527 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:30 smithi090 ceph-mon[30018]: pgmap v53: 33 pgs: 33 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:21:30.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:30 smithi193 ceph-mon[25614]: pgmap v53: 33 pgs: 33 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-16T08:21:32.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:32 smithi193 ceph-mon[25614]: pgmap v54: 33 pgs: 33 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 537 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-16T08:21:32.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:32 smithi090 ceph-mon[30018]: pgmap v54: 33 pgs: 33 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 537 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-16T08:21:32.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:32 smithi090 ceph-mon[23701]: pgmap v54: 33 pgs: 33 active+clean; 578 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 537 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-16T08:21:33.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:33 smithi193 ceph-mon[25614]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T08:21:33.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:33 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3808034929' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-09-16T08:21:33.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:33 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-09-16T08:21:33.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:33 smithi090 ceph-mon[30018]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T08:21:33.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:33 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3808034929' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-09-16T08:21:33.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:33 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-09-16T08:21:33.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:33 smithi090 ceph-mon[23701]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T08:21:33.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:33 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3808034929' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-09-16T08:21:33.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:33 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-09-16T08:21:34.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:34 smithi193 ceph-mon[25614]: pgmap v56: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 921 B/s wr, 4 op/s 2024-09-16T08:21:34.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:34 smithi193 ceph-mon[25614]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:21:34.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:34 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-09-16T08:21:34.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:34 smithi193 ceph-mon[25614]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T08:21:34.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:34 smithi090 ceph-mon[30018]: pgmap v56: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 921 B/s wr, 4 op/s 2024-09-16T08:21:34.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:34 smithi090 ceph-mon[30018]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:21:34.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:34 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-09-16T08:21:34.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:34 smithi090 ceph-mon[30018]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T08:21:34.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:34 smithi090 ceph-mon[23701]: pgmap v56: 65 pgs: 32 unknown, 33 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 921 B/s wr, 4 op/s 2024-09-16T08:21:34.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:34 smithi090 ceph-mon[23701]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:21:34.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:34 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-09-16T08:21:34.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:34 smithi090 ceph-mon[23701]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T08:21:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:35 smithi193 ceph-mon[25614]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T08:21:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:35 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3808034929' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-09-16T08:21:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:35 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-09-16T08:21:35.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:35 smithi090 ceph-mon[30018]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T08:21:35.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:35 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3808034929' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-09-16T08:21:35.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:35 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-09-16T08:21:35.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:35 smithi090 ceph-mon[23701]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T08:21:35.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:35 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3808034929' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-09-16T08:21:35.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:35 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-09-16T08:21:36.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:36 smithi193 ceph-mon[25614]: pgmap v59: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-16T08:21:36.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:36 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-09-16T08:21:36.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:36 smithi193 ceph-mon[25614]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T08:21:36.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:36 smithi090 ceph-mon[30018]: pgmap v59: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-16T08:21:36.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:36 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-09-16T08:21:36.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:36 smithi090 ceph-mon[30018]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T08:21:36.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:36 smithi090 ceph-mon[23701]: pgmap v59: 97 pgs: 64 unknown, 33 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 853 B/s wr, 5 op/s 2024-09-16T08:21:36.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:36 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-09-16T08:21:36.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:36 smithi090 ceph-mon[23701]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T08:21:37.713 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:37 smithi090 ceph-mon[30018]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T08:21:37.713 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:37 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/4155975856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T08:21:37.713 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:37 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T08:21:37.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:37 smithi090 ceph-mon[23701]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T08:21:37.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:37 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/4155975856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T08:21:37.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:37 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T08:21:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:37 smithi193 ceph-mon[25614]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T08:21:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:37 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/4155975856' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T08:21:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:37 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-09-16T08:21:38.449 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:21:38.449 INFO:teuthology.orchestra.run.smithi090.stdout: "id": "d7c44a29-c8de-4c46-8bf0-21fcbe948cc4", 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 1, 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "predecessor_uuid": "fe82523e-e900-4a4e-a386-ac76437cbd2d", 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "sync_status": [], 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "period_map": { 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "id": "d7c44a29-c8de-4c46-8bf0-21fcbe948cc4", 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "zonegroups": [ 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "id": "e7623a06-4b41-4bd9-9c85-b20a608c5a6b", 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default", 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "api_name": "default", 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "is_master": true, 2024-09-16T08:21:38.450 INFO:teuthology.orchestra.run.smithi090.stdout: "endpoints": [], 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "hostnames": [], 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "hostnames_s3website": [], 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "master_zone": "c413f657-35f9-42f4-b76d-6912a5217473", 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "zones": [ 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "id": "c413f657-35f9-42f4-b76d-6912a5217473", 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "z", 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "endpoints": [], 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "log_meta": false, 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "log_data": false, 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "bucket_index_max_shards": 11, 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "read_only": false, 2024-09-16T08:21:38.451 INFO:teuthology.orchestra.run.smithi090.stdout: "tier_type": "", 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: "sync_from_all": true, 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: "sync_from": [], 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_zone": "", 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: "supported_features": [ 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: "compress-encrypted", 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: "resharding" 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: "placement_targets": [ 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-16T08:21:38.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default-placement", 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_classes": [ 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: "STANDARD" 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: "default_placement": "default-placement", 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: "realm_id": "c8d396c0-97e8-49c7-a830-c8c33caf2528", 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: "sync_policy": { 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: "groups": [] 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled_features": [ 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: "resharding" 2024-09-16T08:21:38.453 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: "short_zone_ids": [ 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: "key": "c413f657-35f9-42f4-b76d-6912a5217473", 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 4049810871 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: "master_zonegroup": "e7623a06-4b41-4bd9-9c85-b20a608c5a6b", 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: "master_zone": "c413f657-35f9-42f4-b76d-6912a5217473", 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: "period_config": { 2024-09-16T08:21:38.454 INFO:teuthology.orchestra.run.smithi090.stdout: "bucket_quota": { 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": false, 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "check_on_raw": false, 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max_size": -1, 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max_size_kb": 0, 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max_objects": -1 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "user_quota": { 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": false, 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "check_on_raw": false, 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max_size": -1, 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max_size_kb": 0, 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max_objects": -1 2024-09-16T08:21:38.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "user_ratelimit": { 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max_read_ops": 0, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max_write_ops": 0, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max_read_bytes": 0, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max_write_bytes": 0, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": false 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "bucket_ratelimit": { 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max_read_ops": 0, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max_write_ops": 0, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max_read_bytes": 0, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max_write_bytes": 0, 2024-09-16T08:21:38.456 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": false 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: "anonymous_ratelimit": { 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max_read_ops": 0, 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max_write_ops": 0, 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max_read_bytes": 0, 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max_write_bytes": 0, 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": false 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: "realm_id": "c8d396c0-97e8-49c7-a830-c8c33caf2528", 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: "realm_name": "r", 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout: "realm_epoch": 2 2024-09-16T08:21:38.457 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:21:38.688 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[30018]: pgmap v62: 129 pgs: 96 unknown, 33 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:38.689 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-09-16T08:21:38.689 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[30018]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T08:21:38.689 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/4155975856' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T08:21:38.689 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T08:21:38.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[23701]: pgmap v62: 129 pgs: 96 unknown, 33 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:38.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-09-16T08:21:38.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[23701]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T08:21:38.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/4155975856' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T08:21:38.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:38 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T08:21:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:38 smithi193 ceph-mon[25614]: pgmap v62: 129 pgs: 96 unknown, 33 active+clean; 579 KiB data, 218 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:21:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:38 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-09-16T08:21:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:38 smithi193 ceph-mon[25614]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T08:21:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:38 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/4155975856' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T08:21:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:38 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-16T08:21:39.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000' 2024-09-16T08:21:39.347 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:39.652 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:39 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T08:21:39.653 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:39 smithi090 ceph-mon[30018]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T08:21:39.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:39 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T08:21:39.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:39 smithi090 ceph-mon[23701]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T08:21:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:39 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T08:21:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:39 smithi193 ceph-mon[25614]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T08:21:40.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:39 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:21:39] "GET /metrics HTTP/1.1" 200 28590 "" "Prometheus/2.43.0" 2024-09-16T08:21:40.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:40 smithi193 ceph-mon[25614]: pgmap v65: 129 pgs: 32 creating+peering, 97 active+clean; 579 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:21:40.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:40 smithi193 ceph-mon[25614]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:21:40.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:40 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:21:40.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:40.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:40.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:40 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[30018]: pgmap v65: 129 pgs: 32 creating+peering, 97 active+clean; 579 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[30018]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[23701]: pgmap v65: 129 pgs: 32 creating+peering, 97 active+clean; 579 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 511 B/s wr, 1 op/s 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[23701]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:21:40.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:40.756 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:40.756 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:40 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:41.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:41 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:41.702 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:41 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:41.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:41 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:42.089 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled rgw.foo update... 2024-09-16T08:21:42.699 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: pgmap v66: 129 pgs: 32 creating+peering, 97 active+clean; 579 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 180 B/s rd, 361 B/s wr, 1 op/s 2024-09-16T08:21:42.699 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:42.699 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:42.700 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:42.700 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:21:42.700 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:42.700 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi193.gjyvkw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:21:42.701 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi193.gjyvkw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:21:42.701 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:42.701 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:42.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: pgmap v66: 129 pgs: 32 creating+peering, 97 active+clean; 579 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 180 B/s rd, 361 B/s wr, 1 op/s 2024-09-16T08:21:42.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:42.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:42.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:42.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:21:42.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:42.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi193.gjyvkw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:21:42.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi193.gjyvkw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:21:42.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:42.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:42.746 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: pgmap v66: 129 pgs: 32 creating+peering, 97 active+clean; 579 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 180 B/s rd, 361 B/s wr, 1 op/s 2024-09-16T08:21:42.746 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:42.746 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:42.746 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:42.746 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:21:42.747 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:42.747 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi193.gjyvkw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:21:42.747 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi193.gjyvkw", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:21:42.747 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:42.747 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:42.768 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch apply rgw smpl' 2024-09-16T08:21:43.050 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:43.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:43 smithi193 ceph-mon[25614]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:21:43.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:43 smithi193 ceph-mon[25614]: Saving service rgw.foo spec with placement count:2 2024-09-16T08:21:43.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:43 smithi193 ceph-mon[25614]: Deploying daemon rgw.foo.smithi193.gjyvkw on smithi193 2024-09-16T08:21:43.561 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:43 smithi090 ceph-mon[30018]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:21:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:43 smithi090 ceph-mon[30018]: Saving service rgw.foo spec with placement count:2 2024-09-16T08:21:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:43 smithi090 ceph-mon[30018]: Deploying daemon rgw.foo.smithi193.gjyvkw on smithi193 2024-09-16T08:21:43.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:43 smithi090 ceph-mon[23701]: from='client.24611 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:21:43.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:43 smithi090 ceph-mon[23701]: Saving service rgw.foo spec with placement count:2 2024-09-16T08:21:43.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:43 smithi090 ceph-mon[23701]: Deploying daemon rgw.foo.smithi193.gjyvkw on smithi193 2024-09-16T08:21:44.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:44 smithi193 ceph-mon[25614]: pgmap v67: 129 pgs: 129 active+clean; 581 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-16T08:21:44.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:44 smithi090 ceph-mon[23701]: pgmap v67: 129 pgs: 129 active+clean; 581 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-16T08:21:44.696 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:44 smithi090 ceph-mon[30018]: pgmap v67: 129 pgs: 129 active+clean; 581 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.8 KiB/s wr, 5 op/s 2024-09-16T08:21:45.399 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled rgw.smpl update... 2024-09-16T08:21:46.110 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph osd pool create foo' 2024-09-16T08:21:46.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: pgmap v68: 129 pgs: 129 active+clean; 581 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-16T08:21:46.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:21:46.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: Saving service rgw.smpl spec with placement count:2 2024-09-16T08:21:46.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi090.fbrjws", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:21:46.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi090.fbrjws", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: pgmap v68: 129 pgs: 129 active+clean; 581 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: Saving service rgw.smpl spec with placement count:2 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi090.fbrjws", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:21:46.493 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi090.fbrjws", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:21:46.494 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.494 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:46 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: pgmap v68: 129 pgs: 129 active+clean; 581 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: from='client.24617 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: Saving service rgw.smpl spec with placement count:2 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi090.fbrjws", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi090.fbrjws", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:21:46.596 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:46.597 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:46 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:21:46.939 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:47.726 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:47 smithi090 ceph-mon[30018]: Deploying daemon rgw.foo.smithi090.fbrjws on smithi090 2024-09-16T08:21:47.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:47 smithi090 ceph-mon[23701]: Deploying daemon rgw.foo.smithi090.fbrjws on smithi090 2024-09-16T08:21:47.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:47 smithi193 ceph-mon[25614]: Deploying daemon rgw.foo.smithi090.fbrjws on smithi090 2024-09-16T08:21:48.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:48 smithi090 ceph-mon[23701]: pgmap v69: 129 pgs: 129 active+clean; 581 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-16T08:21:48.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:48 smithi090 ceph-mon[30018]: pgmap v69: 129 pgs: 129 active+clean; 581 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-16T08:21:48.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:48 smithi193 ceph-mon[25614]: pgmap v69: 129 pgs: 129 active+clean; 581 KiB data, 223 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-16T08:21:50.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:49 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:21:49] "GET /metrics HTTP/1.1" 200 35480 "" "Prometheus/2.43.0" 2024-09-16T08:21:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'foo' created 2024-09-16T08:21:50.457 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: pgmap v70: 129 pgs: 129 active+clean; 584 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 4.0 KiB/s wr, 107 op/s 2024-09-16T08:21:50.457 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.458 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.458 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.458 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.458 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.458 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: Saving service rgw.foo spec with placement count:2 2024-09-16T08:21:50.458 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.459 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.459 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:50.459 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1910651924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-09-16T08:21:50.459 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:50 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-09-16T08:21:50.736 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: pgmap v70: 129 pgs: 129 active+clean; 584 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 4.0 KiB/s wr, 107 op/s 2024-09-16T08:21:50.736 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.736 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.736 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.736 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.737 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.737 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: Saving service rgw.foo spec with placement count:2 2024-09-16T08:21:50.737 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.737 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.737 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:50.737 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1910651924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-09-16T08:21:50.737 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-09-16T08:21:50.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: pgmap v70: 129 pgs: 129 active+clean; 584 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 45 KiB/s rd, 4.0 KiB/s wr, 107 op/s 2024-09-16T08:21:50.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: Saving service rgw.foo spec with placement count:2 2024-09-16T08:21:50.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:50.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:21:50.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1910651924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-09-16T08:21:50.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:50 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-09-16T08:21:51.139 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'rbd pool init foo' 2024-09-16T08:21:51.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:51 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-09-16T08:21:51.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:51 smithi193 ceph-mon[25614]: osdmap e73: 8 total, 8 up, 8 in 2024-09-16T08:21:51.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:51 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:51.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:51 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-09-16T08:21:51.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:51 smithi090 ceph-mon[30018]: osdmap e73: 8 total, 8 up, 8 in 2024-09-16T08:21:51.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:51 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:51.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:51 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-09-16T08:21:51.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:51 smithi090 ceph-mon[23701]: osdmap e73: 8 total, 8 up, 8 in 2024-09-16T08:21:51.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:51 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:52.343 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:52.501 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:52 smithi193 ceph-mon[25614]: pgmap v72: 161 pgs: 32 unknown, 129 active+clean; 584 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 4.3 KiB/s wr, 114 op/s 2024-09-16T08:21:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:52 smithi193 ceph-mon[25614]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:21:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:52 smithi193 ceph-mon[25614]: osdmap e74: 8 total, 8 up, 8 in 2024-09-16T08:21:52.774 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:52 smithi090 ceph-mon[30018]: pgmap v72: 161 pgs: 32 unknown, 129 active+clean; 584 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 4.3 KiB/s wr, 114 op/s 2024-09-16T08:21:52.775 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:52 smithi090 ceph-mon[30018]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:21:52.775 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:52 smithi090 ceph-mon[30018]: osdmap e74: 8 total, 8 up, 8 in 2024-09-16T08:21:52.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:52 smithi090 ceph-mon[23701]: pgmap v72: 161 pgs: 32 unknown, 129 active+clean; 584 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 4.3 KiB/s wr, 114 op/s 2024-09-16T08:21:52.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:52 smithi090 ceph-mon[23701]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T08:21:52.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:52 smithi090 ceph-mon[23701]: osdmap e74: 8 total, 8 up, 8 in 2024-09-16T08:21:53.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:53 smithi090 ceph-mon[23701]: osdmap e75: 8 total, 8 up, 8 in 2024-09-16T08:21:53.647 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:53 smithi090 ceph-mon[30018]: osdmap e75: 8 total, 8 up, 8 in 2024-09-16T08:21:53.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:53 smithi193 ceph-mon[25614]: osdmap e75: 8 total, 8 up, 8 in 2024-09-16T08:21:54.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:54 smithi090 ceph-mon[23701]: pgmap v75: 161 pgs: 161 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s rd, 5.7 KiB/s wr, 352 op/s 2024-09-16T08:21:54.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:54 smithi090 ceph-mon[30018]: pgmap v75: 161 pgs: 161 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s rd, 5.7 KiB/s wr, 352 op/s 2024-09-16T08:21:54.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:54 smithi193 ceph-mon[25614]: pgmap v75: 161 pgs: 161 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 185 KiB/s rd, 5.7 KiB/s wr, 352 op/s 2024-09-16T08:21:55.515 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:55 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/858382875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-09-16T08:21:55.516 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:55 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-09-16T08:21:55.516 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:55 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/858382875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-09-16T08:21:55.516 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:55 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-09-16T08:21:55.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:55 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/858382875' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-09-16T08:21:55.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:55 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-09-16T08:21:56.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: pgmap v76: 161 pgs: 161 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s rd, 170 B/s wr, 165 op/s 2024-09-16T08:21:56.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-09-16T08:21:56.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: osdmap e76: 8 total, 8 up, 8 in 2024-09-16T08:21:56.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-16T08:21:56.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-16T08:21:56.702 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: pgmap v76: 161 pgs: 161 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s rd, 170 B/s wr, 165 op/s 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: osdmap e76: 8 total, 8 up, 8 in 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-16T08:21:56.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-16T08:21:56.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-16T08:21:56.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-16T08:21:56.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: pgmap v76: 161 pgs: 161 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s rd, 170 B/s wr, 165 op/s 2024-09-16T08:21:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-09-16T08:21:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: osdmap e76: 8 total, 8 up, 8 in 2024-09-16T08:21:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]: dispatch 2024-09-16T08:21:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-16T08:21:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-16T08:21:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-09-16T08:21:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:21:57.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]': finished 2024-09-16T08:21:57.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-16T08:21:57.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-16T08:21:57.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-16T08:21:57.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:57 smithi193 ceph-mon[25614]: osdmap e77: 8 total, 8 up, 8 in 2024-09-16T08:21:57.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:57.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:57.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]': finished 2024-09-16T08:21:57.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-16T08:21:57.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-16T08:21:57.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-16T08:21:57.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[30018]: osdmap e77: 8 total, 8 up, 8 in 2024-09-16T08:21:57.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:57.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:57.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.3", "id": [7, 4]}]': finished 2024-09-16T08:21:57.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-16T08:21:57.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-16T08:21:57.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-09-16T08:21:57.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[23701]: osdmap e77: 8 total, 8 up, 8 in 2024-09-16T08:21:57.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:57.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:58.481 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch apply iscsi foo u p' 2024-09-16T08:21:58.684 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:21:58.744 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:58 smithi193 ceph-mon[25614]: pgmap v79: 161 pgs: 161 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 181 B/s wr, 176 op/s 2024-09-16T08:21:58.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:58 smithi193 ceph-mon[25614]: osdmap e78: 8 total, 8 up, 8 in 2024-09-16T08:21:58.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:58 smithi193 ceph-mon[25614]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:21:58.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:58 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:21:58.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:58.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:58.745 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:21:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:58.805 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[30018]: pgmap v79: 161 pgs: 161 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 181 B/s wr, 176 op/s 2024-09-16T08:21:58.805 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[30018]: osdmap e78: 8 total, 8 up, 8 in 2024-09-16T08:21:58.805 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[30018]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:21:58.806 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:21:58.806 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:58.806 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:58.806 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:58.806 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[23701]: pgmap v79: 161 pgs: 161 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 114 KiB/s rd, 181 B/s wr, 176 op/s 2024-09-16T08:21:58.806 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[23701]: osdmap e78: 8 total, 8 up, 8 in 2024-09-16T08:21:58.806 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[23701]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T08:21:58.806 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:21:58.806 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:58.807 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:21:58.807 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:21:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:00.051 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:21:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:21:59] "GET /metrics HTTP/1.1" 200 38126 "" "Prometheus/2.43.0" 2024-09-16T08:22:00.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:00 smithi193 ceph-mon[25614]: pgmap v81: 161 pgs: 4 peering, 157 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 170 B/s wr, 0 op/s 2024-09-16T08:22:00.639 INFO:teuthology.orchestra.run.smithi090.stdout:Scheduled iscsi.foo update... 2024-09-16T08:22:00.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:00 smithi090 ceph-mon[23701]: pgmap v81: 161 pgs: 4 peering, 157 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 170 B/s wr, 0 op/s 2024-09-16T08:22:00.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:00 smithi090 ceph-mon[30018]: pgmap v81: 161 pgs: 4 peering, 157 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 170 B/s wr, 0 op/s 2024-09-16T08:22:01.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:00 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:00] ENGINE Bus STOPPING 2024-09-16T08:22:01.290 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'sleep 120' 2024-09-16T08:22:01.561 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:22:01.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE Bus STOPPED 2024-09-16T08:22:01.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE Bus STARTING 2024-09-16T08:22:01.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE Serving on http://:::9283 2024-09-16T08:22:01.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE Bus STARTED 2024-09-16T08:22:01.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE Bus STOPPING 2024-09-16T08:22:01.832 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.832 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.832 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:01.832 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:22:01.832 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: pgmap v82: 161 pgs: 4 peering, 157 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 185 B/s wr, 0 op/s 2024-09-16T08:22:01.832 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: Checking dashboard <-> RGW credentials 2024-09-16T08:22:01.832 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:22:01.832 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: Saving service iscsi.foo spec with placement count:1 2024-09-16T08:22:01.832 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi090.ebelfk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi090.ebelfk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[30018]: Deploying daemon rgw.smpl.smithi090.ebelfk on smithi090 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.833 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: pgmap v82: 161 pgs: 4 peering, 157 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 185 B/s wr, 0 op/s 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: Checking dashboard <-> RGW credentials 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: Saving service iscsi.foo spec with placement count:1 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi090.ebelfk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi090.ebelfk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:01.834 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:01.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-mon[23701]: Deploying daemon rgw.smpl.smithi090.ebelfk on smithi090 2024-09-16T08:22:02.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:02.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:02.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:02.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:22:02.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: pgmap v82: 161 pgs: 4 peering, 157 active+clean; 584 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 185 B/s wr, 0 op/s 2024-09-16T08:22:02.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: Checking dashboard <-> RGW credentials 2024-09-16T08:22:02.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:22:02.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: Saving service iscsi.foo spec with placement count:1 2024-09-16T08:22:02.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:02.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:02.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:02.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:02.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi090.ebelfk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:22:02.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi090.ebelfk", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:22:02.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:02.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:02.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:01 smithi193 ceph-mon[25614]: Deploying daemon rgw.smpl.smithi090.ebelfk on smithi090 2024-09-16T08:22:02.186 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:22:02.186 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE Bus STOPPED 2024-09-16T08:22:02.186 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE Bus STARTING 2024-09-16T08:22:02.187 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE Serving on http://:::9283 2024-09-16T08:22:02.187 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:01 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:01] ENGINE Bus STARTED 2024-09-16T08:22:02.636 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:22:02.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:02 smithi090 ceph-mon[30018]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-16T08:22:02.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:02 smithi090 ceph-mon[23701]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-16T08:22:03.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:02 smithi193 ceph-mon[25614]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-16T08:22:03.650 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:03 smithi090 ceph-mon[30018]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s rd, 574 B/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:03.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:03 smithi090 ceph-mon[23701]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s rd, 574 B/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:04.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:03 smithi193 ceph-mon[25614]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s rd, 574 B/s wr, 4 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:06.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:05 smithi193 ceph-mon[25614]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 503 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:06.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:05 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:05 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:05 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:05 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi193.cgoeeh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:22:06.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:05 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi193.cgoeeh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:22:06.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:05 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:05 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:06.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:05 smithi193 ceph-mon[25614]: Deploying daemon rgw.smpl.smithi193.cgoeeh on smithi193 2024-09-16T08:22:06.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[30018]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 503 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:06.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi193.cgoeeh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:22:06.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi193.cgoeeh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:22:06.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[30018]: Deploying daemon rgw.smpl.smithi193.cgoeeh on smithi193 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[23701]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 3.3 KiB/s rd, 503 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi193.cgoeeh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi193.cgoeeh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:06.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:05 smithi090 ceph-mon[23701]: Deploying daemon rgw.smpl.smithi193.cgoeeh on smithi193 2024-09-16T08:22:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 430 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: osdmap e79: 8 total, 8 up, 8 in 2024-09-16T08:22:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: Saving service rgw.smpl spec with placement count:2 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[23701]: Deploying daemon iscsi.foo.smithi090.jxkvvk on smithi090 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 430 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: osdmap e79: 8 total, 8 up, 8 in 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.177 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.178 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.178 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: Saving service rgw.smpl spec with placement count:2 2024-09-16T08:22:08.178 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.178 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.178 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:22:08.178 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T08:22:08.178 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:08.178 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:08 smithi090 ceph-mon[30018]: Deploying daemon iscsi.foo.smithi090.jxkvvk on smithi090 2024-09-16T08:22:08.314 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 430 B/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:08.314 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: osdmap e79: 8 total, 8 up, 8 in 2024-09-16T08:22:08.314 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.314 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.314 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.314 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: Saving service rgw.smpl spec with placement count:2 2024-09-16T08:22:08.315 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.315 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:08.315 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:22:08.315 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T08:22:08.315 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:08.315 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:08 smithi193 ceph-mon[25614]: Deploying daemon iscsi.foo.smithi090.jxkvvk on smithi090 2024-09-16T08:22:10.051 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:22:09] "GET /metrics HTTP/1.1" 200 38126 "" "Prometheus/2.43.0" 2024-09-16T08:22:10.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:10 smithi193 ceph-mon[25614]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 131 KiB/s rd, 537 B/s wr, 200 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:10.311 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:10 smithi090 ceph-mon[30018]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 131 KiB/s rd, 537 B/s wr, 200 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:10.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:10 smithi090 ceph-mon[23701]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 131 KiB/s rd, 537 B/s wr, 200 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:12.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s rd, 511 B/s wr, 191 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:12.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1232396264' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/689936012' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3621746675"}]: dispatch 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s rd, 511 B/s wr, 191 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:12.607 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.608 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:12.608 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.608 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.608 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.608 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.608 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.608 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:22:12.609 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1232396264' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T08:22:12.609 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:12 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/689936012' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3621746675"}]: dispatch 2024-09-16T08:22:12.614 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s rd, 511 B/s wr, 191 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T08:22:12.614 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.614 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:12.614 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:12.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:22:12.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1232396264' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T08:22:12.615 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:12 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/689936012' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3621746675"}]: dispatch 2024-09-16T08:22:13.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[30018]: Checking pool "foo" exists for service iscsi.foo 2024-09-16T08:22:13.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/689936012' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3621746675"}]': finished 2024-09-16T08:22:13.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[30018]: osdmap e80: 8 total, 8 up, 8 in 2024-09-16T08:22:13.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[30018]: mgrmap e21: y(active, since 2m), standbys: x 2024-09-16T08:22:13.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2468444404' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1992058521"}]: dispatch 2024-09-16T08:22:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[23701]: Checking pool "foo" exists for service iscsi.foo 2024-09-16T08:22:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/689936012' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3621746675"}]': finished 2024-09-16T08:22:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[23701]: osdmap e80: 8 total, 8 up, 8 in 2024-09-16T08:22:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[23701]: mgrmap e21: y(active, since 2m), standbys: x 2024-09-16T08:22:13.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:13 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2468444404' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1992058521"}]: dispatch 2024-09-16T08:22:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:13 smithi193 ceph-mon[25614]: Checking pool "foo" exists for service iscsi.foo 2024-09-16T08:22:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:13 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/689936012' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3621746675"}]': finished 2024-09-16T08:22:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:13 smithi193 ceph-mon[25614]: osdmap e80: 8 total, 8 up, 8 in 2024-09-16T08:22:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:13 smithi193 ceph-mon[25614]: mgrmap e21: y(active, since 2m), standbys: x 2024-09-16T08:22:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:13 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2468444404' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1992058521"}]: dispatch 2024-09-16T08:22:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:14 smithi193 ceph-mon[25614]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 129 KiB/s rd, 307 B/s wr, 200 op/s 2024-09-16T08:22:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:14 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:14 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2468444404' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1992058521"}]': finished 2024-09-16T08:22:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:14 smithi193 ceph-mon[25614]: osdmap e81: 8 total, 8 up, 8 in 2024-09-16T08:22:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:14 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/162406561' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2522025803"}]: dispatch 2024-09-16T08:22:14.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:14 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2522025803"}]: dispatch 2024-09-16T08:22:14.675 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[30018]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 129 KiB/s rd, 307 B/s wr, 200 op/s 2024-09-16T08:22:14.675 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:14.675 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2468444404' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1992058521"}]': finished 2024-09-16T08:22:14.675 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[30018]: osdmap e81: 8 total, 8 up, 8 in 2024-09-16T08:22:14.675 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/162406561' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2522025803"}]: dispatch 2024-09-16T08:22:14.675 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2522025803"}]: dispatch 2024-09-16T08:22:14.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[23701]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 129 KiB/s rd, 307 B/s wr, 200 op/s 2024-09-16T08:22:14.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:14.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2468444404' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1992058521"}]': finished 2024-09-16T08:22:14.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[23701]: osdmap e81: 8 total, 8 up, 8 in 2024-09-16T08:22:14.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/162406561' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2522025803"}]: dispatch 2024-09-16T08:22:14.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:14 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2522025803"}]: dispatch 2024-09-16T08:22:15.795 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:15 smithi090 ceph-mon[30018]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 172 KiB/s rd, 408 B/s wr, 266 op/s 2024-09-16T08:22:15.795 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:15 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2522025803"}]': finished 2024-09-16T08:22:15.795 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:15 smithi090 ceph-mon[30018]: osdmap e82: 8 total, 8 up, 8 in 2024-09-16T08:22:15.795 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:15 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1674781120' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2556824909"}]: dispatch 2024-09-16T08:22:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:15 smithi090 ceph-mon[23701]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 172 KiB/s rd, 408 B/s wr, 266 op/s 2024-09-16T08:22:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:15 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2522025803"}]': finished 2024-09-16T08:22:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:15 smithi090 ceph-mon[23701]: osdmap e82: 8 total, 8 up, 8 in 2024-09-16T08:22:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:15 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1674781120' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2556824909"}]: dispatch 2024-09-16T08:22:15.888 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:15 smithi193 ceph-mon[25614]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 172 KiB/s rd, 408 B/s wr, 266 op/s 2024-09-16T08:22:15.889 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:15 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2522025803"}]': finished 2024-09-16T08:22:15.889 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:15 smithi193 ceph-mon[25614]: osdmap e82: 8 total, 8 up, 8 in 2024-09-16T08:22:15.889 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:15 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1674781120' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2556824909"}]: dispatch 2024-09-16T08:22:16.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:16 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1674781120' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2556824909"}]': finished 2024-09-16T08:22:16.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:16 smithi090 ceph-mon[30018]: osdmap e83: 8 total, 8 up, 8 in 2024-09-16T08:22:16.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:16 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/134221968' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2766943845"}]: dispatch 2024-09-16T08:22:16.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:16 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:16.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:16 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1674781120' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2556824909"}]': finished 2024-09-16T08:22:16.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:16 smithi090 ceph-mon[23701]: osdmap e83: 8 total, 8 up, 8 in 2024-09-16T08:22:16.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:16 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/134221968' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2766943845"}]: dispatch 2024-09-16T08:22:16.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:16 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.001 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:16 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1674781120' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2556824909"}]': finished 2024-09-16T08:22:17.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:16 smithi193 ceph-mon[25614]: osdmap e83: 8 total, 8 up, 8 in 2024-09-16T08:22:17.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:16 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/134221968' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2766943845"}]: dispatch 2024-09-16T08:22:17.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:16 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.962 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:22:17.962 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/134221968' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2766943845"}]': finished 2024-09-16T08:22:17.962 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: osdmap e84: 8 total, 8 up, 8 in 2024-09-16T08:22:17.962 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/4053115192' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/1569791063"}]: dispatch 2024-09-16T08:22:17.962 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.962 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.962 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.962 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.962 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.963 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.963 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.963 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:17.963 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:17 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:22:18.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/134221968' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2766943845"}]': finished 2024-09-16T08:22:18.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: osdmap e84: 8 total, 8 up, 8 in 2024-09-16T08:22:18.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/4053115192' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/1569791063"}]: dispatch 2024-09-16T08:22:18.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.065 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:22:18.065 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/134221968' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2766943845"}]': finished 2024-09-16T08:22:18.065 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: osdmap e84: 8 total, 8 up, 8 in 2024-09-16T08:22:18.065 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/4053115192' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/1569791063"}]: dispatch 2024-09-16T08:22:18.066 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.066 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.066 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.066 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.067 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.067 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.067 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.067 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.067 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:17 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:18.927 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:18 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/4053115192' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/1569791063"}]': finished 2024-09-16T08:22:18.927 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:18 smithi193 ceph-mon[25614]: osdmap e85: 8 total, 8 up, 8 in 2024-09-16T08:22:18.927 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:18 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1125703166' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/688775638"}]: dispatch 2024-09-16T08:22:19.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:18 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/4053115192' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/1569791063"}]': finished 2024-09-16T08:22:19.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:18 smithi090 ceph-mon[23701]: osdmap e85: 8 total, 8 up, 8 in 2024-09-16T08:22:19.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:18 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1125703166' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/688775638"}]: dispatch 2024-09-16T08:22:19.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:18 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/4053115192' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/1569791063"}]': finished 2024-09-16T08:22:19.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:18 smithi090 ceph-mon[30018]: osdmap e85: 8 total, 8 up, 8 in 2024-09-16T08:22:19.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:18 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1125703166' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/688775638"}]: dispatch 2024-09-16T08:22:20.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:19 smithi193 ceph-mon[25614]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:20.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:19 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1125703166' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/688775638"}]': finished 2024-09-16T08:22:20.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:19 smithi193 ceph-mon[25614]: osdmap e86: 8 total, 8 up, 8 in 2024-09-16T08:22:20.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:19 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/31217996' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/688775638"}]: dispatch 2024-09-16T08:22:20.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:19 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/688775638"}]: dispatch 2024-09-16T08:22:20.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:22:19] "GET /metrics HTTP/1.1" 200 38463 "" "Prometheus/2.43.0" 2024-09-16T08:22:20.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[30018]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:20.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1125703166' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/688775638"}]': finished 2024-09-16T08:22:20.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[30018]: osdmap e86: 8 total, 8 up, 8 in 2024-09-16T08:22:20.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/31217996' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/688775638"}]: dispatch 2024-09-16T08:22:20.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/688775638"}]: dispatch 2024-09-16T08:22:20.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[23701]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:20.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1125703166' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/688775638"}]': finished 2024-09-16T08:22:20.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[23701]: osdmap e86: 8 total, 8 up, 8 in 2024-09-16T08:22:20.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/31217996' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/688775638"}]: dispatch 2024-09-16T08:22:20.064 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:19 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/688775638"}]: dispatch 2024-09-16T08:22:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/688775638"}]': finished 2024-09-16T08:22:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: osdmap e87: 8 total, 8 up, 8 in 2024-09-16T08:22:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1241273603' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1172167221"}]: dispatch 2024-09-16T08:22:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1172167221"}]: dispatch 2024-09-16T08:22:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:20.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:20.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:22:20.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:22:20.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:22:20.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: Checking dashboard <-> RGW credentials 2024-09-16T08:22:20.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:20 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:20.892 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/688775638"}]': finished 2024-09-16T08:22:20.893 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: osdmap e87: 8 total, 8 up, 8 in 2024-09-16T08:22:20.893 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1241273603' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1172167221"}]: dispatch 2024-09-16T08:22:20.893 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1172167221"}]: dispatch 2024-09-16T08:22:20.893 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:20.893 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:20.894 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:20.894 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:22:20.894 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:22:20.894 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:22:20.894 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: Checking dashboard <-> RGW credentials 2024-09-16T08:22:20.894 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:20.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/688775638"}]': finished 2024-09-16T08:22:20.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: osdmap e87: 8 total, 8 up, 8 in 2024-09-16T08:22:20.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1241273603' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1172167221"}]: dispatch 2024-09-16T08:22:20.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1172167221"}]: dispatch 2024-09-16T08:22:20.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:20.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:20.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:20.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:22:20.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:22:20.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 252 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:22:20.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: Checking dashboard <-> RGW credentials 2024-09-16T08:22:20.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:20.898 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:20] ENGINE Bus STOPPING 2024-09-16T08:22:20.898 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:20] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:22:20.898 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:20] ENGINE Bus STOPPED 2024-09-16T08:22:20.898 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:20] ENGINE Bus STARTING 2024-09-16T08:22:20.898 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:20] ENGINE Serving on http://:::9283 2024-09-16T08:22:20.898 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:20 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:22:20] ENGINE Bus STARTED 2024-09-16T08:22:22.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1172167221"}]': finished 2024-09-16T08:22:22.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: osdmap e88: 8 total, 8 up, 8 in 2024-09-16T08:22:22.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-16T08:22:22.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-16T08:22:22.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: Adding iSCSI gateway http://:@172.21.15.90:5000 to Dashboard 2024-09-16T08:22:22.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-16T08:22:22.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-16T08:22:22.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:22.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi090"}]: dispatch 2024-09-16T08:22:22.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi090"}]: dispatch 2024-09-16T08:22:22.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:22.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:22:22.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/4246568052' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/1569791063"}]: dispatch 2024-09-16T08:22:22.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:21 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/1569791063"}]: dispatch 2024-09-16T08:22:22.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1172167221"}]': finished 2024-09-16T08:22:22.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: osdmap e88: 8 total, 8 up, 8 in 2024-09-16T08:22:22.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-16T08:22:22.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-16T08:22:22.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: Adding iSCSI gateway http://:@172.21.15.90:5000 to Dashboard 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi090"}]: dispatch 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi090"}]: dispatch 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/4246568052' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/1569791063"}]: dispatch 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/1569791063"}]: dispatch 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1172167221"}]': finished 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: osdmap e88: 8 total, 8 up, 8 in 2024-09-16T08:22:22.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: Adding iSCSI gateway http://:@172.21.15.90:5000 to Dashboard 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi090"}]: dispatch 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi090"}]: dispatch 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/4246568052' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/1569791063"}]: dispatch 2024-09-16T08:22:22.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:21 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/1569791063"}]: dispatch 2024-09-16T08:22:23.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:22 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:23.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:22 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/1569791063"}]': finished 2024-09-16T08:22:23.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:22 smithi193 ceph-mon[25614]: osdmap e89: 8 total, 8 up, 8 in 2024-09-16T08:22:23.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:22 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1673219440' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3342887308"}]: dispatch 2024-09-16T08:22:23.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:22 smithi193 ceph-mon[25614]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 5.8 KiB/s rd, 5 op/s 2024-09-16T08:22:23.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:23.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/1569791063"}]': finished 2024-09-16T08:22:23.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[23701]: osdmap e89: 8 total, 8 up, 8 in 2024-09-16T08:22:23.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1673219440' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3342887308"}]: dispatch 2024-09-16T08:22:23.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[23701]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 5.8 KiB/s rd, 5 op/s 2024-09-16T08:22:23.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:23.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/1569791063"}]': finished 2024-09-16T08:22:23.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[30018]: osdmap e89: 8 total, 8 up, 8 in 2024-09-16T08:22:23.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1673219440' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3342887308"}]: dispatch 2024-09-16T08:22:23.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:22 smithi090 ceph-mon[30018]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 5.8 KiB/s rd, 5 op/s 2024-09-16T08:22:24.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:23 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1673219440' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3342887308"}]': finished 2024-09-16T08:22:24.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:23 smithi193 ceph-mon[25614]: osdmap e90: 8 total, 8 up, 8 in 2024-09-16T08:22:24.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:23 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/4113699675' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3754108410"}]: dispatch 2024-09-16T08:22:24.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:23 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3754108410"}]: dispatch 2024-09-16T08:22:24.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:23 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1673219440' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3342887308"}]': finished 2024-09-16T08:22:24.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:23 smithi090 ceph-mon[23701]: osdmap e90: 8 total, 8 up, 8 in 2024-09-16T08:22:24.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:23 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/4113699675' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3754108410"}]: dispatch 2024-09-16T08:22:24.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:23 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3754108410"}]: dispatch 2024-09-16T08:22:24.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:23 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1673219440' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3342887308"}]': finished 2024-09-16T08:22:24.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:23 smithi090 ceph-mon[30018]: osdmap e90: 8 total, 8 up, 8 in 2024-09-16T08:22:24.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:23 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/4113699675' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3754108410"}]: dispatch 2024-09-16T08:22:24.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:23 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3754108410"}]: dispatch 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3754108410"}]': finished 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: osdmap e91: 8 total, 8 up, 8 in 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3149035623' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/2155974574"}]: dispatch 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 5 op/s 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 6.1 KiB/s rd, 5 op/s 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: Checking dashboard <-> RGW credentials 2024-09-16T08:22:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:24 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:25.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3754108410"}]': finished 2024-09-16T08:22:25.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: osdmap e91: 8 total, 8 up, 8 in 2024-09-16T08:22:25.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3149035623' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/2155974574"}]: dispatch 2024-09-16T08:22:25.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:25.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:25.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:25.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:22:25.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 5 op/s 2024-09-16T08:22:25.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 6.1 KiB/s rd, 5 op/s 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: Checking dashboard <-> RGW credentials 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/3754108410"}]': finished 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: osdmap e91: 8 total, 8 up, 8 in 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3149035623' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/2155974574"}]: dispatch 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 5.7 KiB/s rd, 5 op/s 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 6.1 KiB/s rd, 5 op/s 2024-09-16T08:22:25.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: Checking dashboard <-> RGW credentials 2024-09-16T08:22:25.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:24 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:26.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:25 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3149035623' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/2155974574"}]': finished 2024-09-16T08:22:26.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:25 smithi090 ceph-mon[30018]: osdmap e92: 8 total, 8 up, 8 in 2024-09-16T08:22:26.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:25 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3136272918' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1806888503"}]: dispatch 2024-09-16T08:22:26.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:25 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1806888503"}]: dispatch 2024-09-16T08:22:26.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:25 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3149035623' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/2155974574"}]': finished 2024-09-16T08:22:26.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:25 smithi090 ceph-mon[23701]: osdmap e92: 8 total, 8 up, 8 in 2024-09-16T08:22:26.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:25 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3136272918' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1806888503"}]: dispatch 2024-09-16T08:22:26.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:25 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1806888503"}]: dispatch 2024-09-16T08:22:26.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:25 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3149035623' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/2155974574"}]': finished 2024-09-16T08:22:26.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:25 smithi193 ceph-mon[25614]: osdmap e92: 8 total, 8 up, 8 in 2024-09-16T08:22:26.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:25 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3136272918' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1806888503"}]: dispatch 2024-09-16T08:22:26.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:25 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1806888503"}]: dispatch 2024-09-16T08:22:27.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1806888503"}]': finished 2024-09-16T08:22:27.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[30018]: osdmap e93: 8 total, 8 up, 8 in 2024-09-16T08:22:27.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1249910775' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/392352616"}]: dispatch 2024-09-16T08:22:27.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[30018]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:22:27.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:27.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1806888503"}]': finished 2024-09-16T08:22:27.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[23701]: osdmap e93: 8 total, 8 up, 8 in 2024-09-16T08:22:27.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1249910775' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/392352616"}]: dispatch 2024-09-16T08:22:27.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[23701]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:22:27.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:26 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:27.153 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:26 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/1806888503"}]': finished 2024-09-16T08:22:27.153 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:26 smithi193 ceph-mon[25614]: osdmap e93: 8 total, 8 up, 8 in 2024-09-16T08:22:27.153 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:26 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1249910775' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/392352616"}]: dispatch 2024-09-16T08:22:27.153 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:26 smithi193 ceph-mon[25614]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:22:27.153 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:26 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:28.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:27 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1249910775' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/392352616"}]': finished 2024-09-16T08:22:28.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:27 smithi193 ceph-mon[25614]: osdmap e94: 8 total, 8 up, 8 in 2024-09-16T08:22:28.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:27 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1153361758' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/2155974574"}]: dispatch 2024-09-16T08:22:28.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:27 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1249910775' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/392352616"}]': finished 2024-09-16T08:22:28.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:27 smithi090 ceph-mon[30018]: osdmap e94: 8 total, 8 up, 8 in 2024-09-16T08:22:28.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:27 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1153361758' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/2155974574"}]: dispatch 2024-09-16T08:22:28.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:27 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1249910775' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/392352616"}]': finished 2024-09-16T08:22:28.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:27 smithi090 ceph-mon[23701]: osdmap e94: 8 total, 8 up, 8 in 2024-09-16T08:22:28.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:27 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1153361758' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/2155974574"}]: dispatch 2024-09-16T08:22:29.036 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:28 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1153361758' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/2155974574"}]': finished 2024-09-16T08:22:29.036 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:28 smithi193 ceph-mon[25614]: osdmap e95: 8 total, 8 up, 8 in 2024-09-16T08:22:29.036 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:28 smithi193 ceph-mon[25614]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 5.5 KiB/s rd, 5 op/s 2024-09-16T08:22:29.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:28 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1153361758' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/2155974574"}]': finished 2024-09-16T08:22:29.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:28 smithi090 ceph-mon[30018]: osdmap e95: 8 total, 8 up, 8 in 2024-09-16T08:22:29.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:28 smithi090 ceph-mon[30018]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 5.5 KiB/s rd, 5 op/s 2024-09-16T08:22:29.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:28 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1153361758' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/2155974574"}]': finished 2024-09-16T08:22:29.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:28 smithi090 ceph-mon[23701]: osdmap e95: 8 total, 8 up, 8 in 2024-09-16T08:22:29.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:28 smithi090 ceph-mon[23701]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 5.5 KiB/s rd, 5 op/s 2024-09-16T08:22:30.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:29 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:22:29] "GET /metrics HTTP/1.1" 200 38501 "" "Prometheus/2.43.0" 2024-09-16T08:22:31.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:31 smithi193 ceph-mon[25614]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 3 op/s 2024-09-16T08:22:31.729 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:31 smithi090 ceph-mon[30018]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 3 op/s 2024-09-16T08:22:31.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:31 smithi090 ceph-mon[23701]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 3 op/s 2024-09-16T08:22:33.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:33 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:33.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:33 smithi090 ceph-mon[23701]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 3 op/s 2024-09-16T08:22:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:33 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:33 smithi090 ceph-mon[30018]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 3 op/s 2024-09-16T08:22:33.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:33 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:33.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:33 smithi193 ceph-mon[25614]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 3 op/s 2024-09-16T08:22:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:35 smithi193 ceph-mon[25614]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 3 op/s 2024-09-16T08:22:35.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:35 smithi090 ceph-mon[23701]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 3 op/s 2024-09-16T08:22:35.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:35 smithi090 ceph-mon[30018]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 3.4 KiB/s rd, 3 op/s 2024-09-16T08:22:37.615 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:37 smithi090 ceph-mon[30018]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 2 op/s 2024-09-16T08:22:37.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:37 smithi090 ceph-mon[23701]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 2 op/s 2024-09-16T08:22:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:37 smithi193 ceph-mon[25614]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 2 op/s 2024-09-16T08:22:39.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:38 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:39.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:38 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:39.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:38 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:39.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:38 smithi193 ceph-mon[25614]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 976 B/s rd, 0 op/s 2024-09-16T08:22:39.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:38 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:39.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:38 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:39.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:38 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:39.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:38 smithi090 ceph-mon[23701]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 976 B/s rd, 0 op/s 2024-09-16T08:22:39.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:38 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:39.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:38 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:39.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:38 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:39.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:38 smithi090 ceph-mon[30018]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 976 B/s rd, 0 op/s 2024-09-16T08:22:40.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:39 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:22:39] "GET /metrics HTTP/1.1" 200 38501 "" "Prometheus/2.43.0" 2024-09-16T08:22:41.635 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:41 smithi193 ceph-mon[25614]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:41.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:41 smithi090 ceph-mon[23701]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:41.741 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:41 smithi090 ceph-mon[30018]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:42.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:42.811 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:42.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:43.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:43 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:43.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:43 smithi193 ceph-mon[25614]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:43.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:43 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:43.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:43 smithi090 ceph-mon[30018]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:43.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:43 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:43.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:43 smithi090 ceph-mon[23701]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:45.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:45 smithi193 ceph-mon[25614]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:45.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:45 smithi090 ceph-mon[30018]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:45.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:45 smithi090 ceph-mon[23701]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:47.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:47 smithi193 ceph-mon[25614]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:47.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:47 smithi090 ceph-mon[30018]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:47.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:47 smithi090 ceph-mon[23701]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:49.620 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:49 smithi193 ceph-mon[25614]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:49.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:49 smithi090 ceph-mon[30018]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:49.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:49 smithi090 ceph-mon[23701]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:50.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:49 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:22:49] "GET /metrics HTTP/1.1" 200 38506 "" "Prometheus/2.43.0" 2024-09-16T08:22:51.428 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:51 smithi193 ceph-mon[25614]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:51.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:51 smithi090 ceph-mon[23701]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:51.751 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:51 smithi090 ceph-mon[30018]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:53.559 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:53 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:53.559 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:53 smithi090 ceph-mon[30018]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:53.559 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:53 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:53.559 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:53 smithi090 ceph-mon[23701]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:53.751 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:53 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:22:53.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:53 smithi193 ceph-mon[25614]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:55.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:55 smithi193 ceph-mon[25614]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:55.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:55 smithi090 ceph-mon[23701]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:55.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:55 smithi090 ceph-mon[30018]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:56.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:56.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:22:57.652 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:57 smithi193 ceph-mon[25614]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:57.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:57 smithi090 ceph-mon[30018]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:57.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:57 smithi090 ceph-mon[23701]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:22:59.001 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:59.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:59.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:58 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:59.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:22:58 smithi193 ceph-mon[25614]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:59.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:59.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:59.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:58 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:59.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:22:58 smithi090 ceph-mon[30018]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:22:59.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:59.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:59.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:58 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:22:59.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:22:58 smithi090 ceph-mon[23701]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:00.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:22:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:22:59] "GET /metrics HTTP/1.1" 200 38504 "" "Prometheus/2.43.0" 2024-09-16T08:23:01.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:01 smithi193 ceph-mon[25614]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:01.754 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:01 smithi090 ceph-mon[30018]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:01.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:01 smithi090 ceph-mon[23701]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:03.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:03 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:03.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:03 smithi193 ceph-mon[25614]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:03.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:03 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:03.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:03 smithi090 ceph-mon[30018]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:03.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:03 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:03.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:03 smithi090 ceph-mon[23701]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:05.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:05 smithi193 ceph-mon[25614]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:05.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:05 smithi090 ceph-mon[23701]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:05.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:05 smithi090 ceph-mon[30018]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:07.615 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:07 smithi090 ceph-mon[30018]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:07.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:07 smithi090 ceph-mon[23701]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:07 smithi193 ceph-mon[25614]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:09.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:09 smithi193 ceph-mon[25614]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:09.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:09 smithi090 ceph-mon[23701]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:09.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:09 smithi090 ceph-mon[30018]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:10.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:23:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:23:09] "GET /metrics HTTP/1.1" 200 38504 "" "Prometheus/2.43.0" 2024-09-16T08:23:11.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:11 smithi193 ceph-mon[25614]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:11.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:11 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:11.761 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:11 smithi090 ceph-mon[23701]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:11.761 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:11 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:11.761 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:11 smithi090 ceph-mon[30018]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:11.761 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:11 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:13.359 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:13 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:13.360 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:13 smithi090 ceph-mon[30018]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:13.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:13 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:13.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:13 smithi090 ceph-mon[23701]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:13 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:13 smithi193 ceph-mon[25614]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:15.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:15 smithi090 ceph-mon[30018]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:15.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:15 smithi090 ceph-mon[23701]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:15.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:15 smithi193 ceph-mon[25614]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:17.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:17 smithi193 ceph-mon[25614]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:17.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:17 smithi090 ceph-mon[23701]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:17.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:17 smithi090 ceph-mon[30018]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:19.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:19.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:19.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:18 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:19.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:18 smithi193 ceph-mon[25614]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:19.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:19.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:19.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:18 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:19.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:18 smithi090 ceph-mon[23701]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:19.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:19.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:19.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:18 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:19.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:18 smithi090 ceph-mon[30018]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:20.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:23:19 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:23:19] "GET /metrics HTTP/1.1" 200 38506 "" "Prometheus/2.43.0" 2024-09-16T08:23:21.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:21 smithi193 ceph-mon[25614]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:21.771 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:21 smithi090 ceph-mon[30018]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:21.772 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:21 smithi090 ceph-mon[23701]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:23.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:23 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:23.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:23 smithi193 ceph-mon[25614]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:23.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:23 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:23.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:23 smithi090 ceph-mon[30018]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:23.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:23 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:23.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:23 smithi090 ceph-mon[23701]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:25.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:25 smithi193 ceph-mon[25614]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:25.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:25 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:23:25.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:25 smithi090 ceph-mon[30018]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:25.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:25 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:23:25.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:25 smithi090 ceph-mon[23701]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:25.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:25 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:23:26.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:26 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:26.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:26 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:26.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:26 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:27.522 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:27 smithi193 ceph-mon[25614]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:27.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:27 smithi090 ceph-mon[23701]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:27.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:27 smithi090 ceph-mon[30018]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:29.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:28 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:29.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:28 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:29.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:28 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:23:29.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:28 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:23:29.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:28 smithi193 ceph-mon[25614]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:23:29.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:28 smithi193 ceph-mon[25614]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:29.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:28 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:29.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:29.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:29.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:23:29.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:23:29.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[30018]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:23:29.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[30018]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:29.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:29.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:29.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:29.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:23:29.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:23:29.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[23701]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:23:29.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[23701]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:29.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:28 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:30.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:23:29 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:23:29] "GET /metrics HTTP/1.1" 200 38506 "" "Prometheus/2.43.0" 2024-09-16T08:23:31.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:31 smithi090 ceph-mon[23701]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:31.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:31 smithi090 ceph-mon[30018]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:31.674 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:31 smithi193 ceph-mon[25614]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:33 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:33 smithi193 ceph-mon[25614]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:33 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:33 smithi090 ceph-mon[30018]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:33.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:33 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:33.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:33 smithi090 ceph-mon[23701]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:35.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:35 smithi090 ceph-mon[23701]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:35.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:35 smithi090 ceph-mon[30018]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:35 smithi193 ceph-mon[25614]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:37.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:37 smithi090 ceph-mon[30018]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:37.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:37 smithi090 ceph-mon[23701]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:37 smithi193 ceph-mon[25614]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:23:39.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:39 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:39.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:39 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:39.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:39 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:39.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:39 smithi193 ceph-mon[25614]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:23:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:39 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:39 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:39 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:39 smithi090 ceph-mon[23701]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:23:39.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:39 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:39 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:39 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:39 smithi090 ceph-mon[30018]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:23:40.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:23:39 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:23:39] "GET /metrics HTTP/1.1" 200 38506 "" "Prometheus/2.43.0" 2024-09-16T08:23:41.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:41 smithi090 ceph-mon[30018]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:41.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:41 smithi090 ceph-mon[23701]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:41.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:41 smithi193 ceph-mon[25614]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:42.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:42.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:42.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:43.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:43 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:43.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:43 smithi090 ceph-mon[23701]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:43 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:43 smithi090 ceph-mon[30018]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:43.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:43 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:43.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:43 smithi193 ceph-mon[25614]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:45.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:45 smithi090 ceph-mon[23701]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:45.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:45 smithi090 ceph-mon[30018]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:45.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:45 smithi193 ceph-mon[25614]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:47.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:47 smithi090 ceph-mon[23701]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:47.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:47 smithi090 ceph-mon[30018]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:47.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:47 smithi193 ceph-mon[25614]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:49.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:49 smithi090 ceph-mon[30018]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:49.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:49 smithi090 ceph-mon[23701]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:49.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:49 smithi193 ceph-mon[25614]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:50.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:23:49 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:23:49] "GET /metrics HTTP/1.1" 200 38505 "" "Prometheus/2.43.0" 2024-09-16T08:23:51.433 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:51 smithi193 ceph-mon[25614]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:51.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:51 smithi090 ceph-mon[30018]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:51.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:51 smithi090 ceph-mon[23701]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:53.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:53 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:53.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:53 smithi193 ceph-mon[25614]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:53.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:53 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:53.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:53 smithi090 ceph-mon[30018]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:53.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:53 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:23:53.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:53 smithi090 ceph-mon[23701]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:55.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:55 smithi193 ceph-mon[25614]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:55.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:55 smithi090 ceph-mon[30018]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:55.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:55 smithi090 ceph-mon[23701]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:57 smithi090 ceph-mon[23701]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:57.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:57 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:57 smithi090 ceph-mon[30018]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:57.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:57 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:57.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:57 smithi193 ceph-mon[25614]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:23:57.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:57 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:23:59.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:59 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:59.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:59 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:59.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:59 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:59.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:23:59 smithi193 ceph-mon[25614]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:59.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:59 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:59.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:59 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:59.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:59 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:59.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:23:59 smithi090 ceph-mon[30018]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:23:59.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:59 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:59.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:59 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:59.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:59 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:23:59.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:23:59 smithi090 ceph-mon[23701]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:00.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:23:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:23:59] "GET /metrics HTTP/1.1" 200 38508 "" "Prometheus/2.43.0" 2024-09-16T08:24:01.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:01 smithi090 ceph-mon[30018]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:01.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:01 smithi090 ceph-mon[23701]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:01.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:01 smithi193 ceph-mon[25614]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:03.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:03 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:03.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:03 smithi193 ceph-mon[25614]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:03.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:03 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:03.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:03 smithi090 ceph-mon[30018]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:03.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:03 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:03.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:03 smithi090 ceph-mon[23701]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:04.958 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-16T08:24:05.171 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:24:05.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:05 smithi193 ceph-mon[25614]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:05.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:05 smithi090 ceph-mon[30018]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:05.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:05 smithi090 ceph-mon[23701]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:07.285 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:07 smithi090 ceph-mon[23701]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:07.534 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:07 smithi090 ceph-mon[30018]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:07.536 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-16T08:24:07.741 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:24:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:07 smithi193 ceph-mon[25614]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:09.377 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:09 smithi090 ceph-mon[30018]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:09.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:09 smithi090 ceph-mon[23701]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:09.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:09 smithi193 ceph-mon[25614]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:10.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:24:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:24:09] "GET /metrics HTTP/1.1" 200 38508 "" "Prometheus/2.43.0" 2024-09-16T08:24:10.121 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-16T08:24:10.322 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:24:11.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:11 smithi090 ceph-mon[23701]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:11.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:11 smithi090 ceph-mon[30018]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:11.621 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:11 smithi193 ceph-mon[25614]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:12.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:12 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:12.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:12 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:12.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:12 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:12.641 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-09-16T08:24:12.845 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:24:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:13 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:13.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:13 smithi090 ceph-mon[23701]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:13 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:13.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:13 smithi090 ceph-mon[30018]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:13.599 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:13 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:13.600 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:13 smithi193 ceph-mon[25614]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:14.509 INFO:teuthology.orchestra.run.smithi090.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:24:15.055 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T08:24:15.069 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-09-16T08:24:15.069 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-09-16T08:24:15.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[30018]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:15.885 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[30018]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:15.886 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[30018]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:24:15.886 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:15.886 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:24:15.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[23701]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:15.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[23701]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:15.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[23701]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:24:15.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:15.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:15 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:24:16.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:15 smithi193 ceph-mon[25614]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:16.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:15 smithi193 ceph-mon[25614]: from='client.24986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:16.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:15 smithi193 ceph-mon[25614]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:24:16.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:15 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:16.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:15 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:24:16.631 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:24:17.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:17 smithi090 ceph-mon[30018]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:17.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:17 smithi090 ceph-mon[23701]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:18.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:17 smithi193 ceph-mon[25614]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:19.173 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-09-16T08:24:19.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:24:19.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:24:19.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:24:19.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:24:19.175 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:24:19.176 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:19.176 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.176 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:24:19.176 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.176 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.176 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:24:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:24:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:24:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:24:19.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:19.770 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:24:19.770 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi090 smithi090 running 1s ago 9m - - 2024-09-16T08:24:19.770 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi193 smithi193 error 1s ago 8m - - 2024-09-16T08:24:19.770 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.a smithi090 *:9093,9094 running (3m) 1s ago 4m 28.7M - 0.25.0 c8568f914cd2 3c66ea902fdc 2024-09-16T08:24:19.770 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.a smithi193 *:3000 running (3m) 1s ago 3m 87.6M - 9.4.7 954c08fa6188 d2a25f536ef6 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:iscsi.foo.smithi090.jxkvvk smithi090 *:5000 running (2m) 1s ago 2m 53.1M - 3.8 fd1d6ca2b4a5 f7eb9aafebf0 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi193 *:8443,8765 running (8m) 1s ago 8m 439M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 4b116b0d4a11 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi090 *:9283,8765 running (9m) 1s ago 9m 501M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 92b236ae51c0 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (9m) 1s ago 10m 51.0M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 87c2b7a329d6 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi193 running (8m) 1s ago 8m 41.4M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a45dcb4d7793 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi090 running (8m) 1s ago 8m 40.6M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 fddfc4718a9c 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.a smithi090 *:9100 running (3m) 1s ago 3m 16.9M - 1.5.0 0da6a335fe13 4a6ade533b17 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.b smithi193 *:9100 running (3m) 1s ago 3m 16.0M - 1.5.0 0da6a335fe13 46a72e78b0a2 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (7m) 1s ago 7m 51.5M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 d8ac704cb999 2024-09-16T08:24:19.771 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (7m) 1s ago 7m 58.0M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0703646c72eb 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (7m) 1s ago 7m 47.6M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3f0553bc3456 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (6m) 1s ago 6m 58.2M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90097820090a 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi193 running (6m) 1s ago 6m 51.4M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 389bfe865f47 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi193 running (5m) 1s ago 5m 49.1M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 b5fa3f6ab4bf 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi193 running (5m) 1s ago 5m 59.7M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 dc9317ffcb0d 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi193 running (5m) 1s ago 5m 59.8M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 9a7223ca10b1 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.a smithi193 *:9095 running (3m) 1s ago 4m 49.7M - 2.43.0 a07b618ecd1d 0416160d55ca 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi090.fbrjws smithi090 *:8000 running (2m) 1s ago 2m 76.1M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 6bd392324db2 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi193.gjyvkw smithi193 *:8000 running (2m) 1s ago 2m 75.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 351896279a6f 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi090.ebelfk smithi090 *:80 running (2m) 1s ago 2m 75.6M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 374e9c0e942b 2024-09-16T08:24:19.772 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi193.cgoeeh smithi193 *:80 running (2m) 1s ago 2m 76.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cf9d24a65b38 2024-09-16T08:24:20.041 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:24:19 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:24:19] "GET /metrics HTTP/1.1" 200 38506 "" "Prometheus/2.43.0" 2024-09-16T08:24:20.041 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:20 smithi090 ceph-mon[23701]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:20.041 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:20 smithi090 ceph-mon[23701]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 3 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 8 2024-09-16T08:24:20.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:24:20.125 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw": { 2024-09-16T08:24:20.125 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:24:20.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:24:20.125 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-09-16T08:24:20.125 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 17 2024-09-16T08:24:20.125 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:24:20.125 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:24:20.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:20 smithi090 ceph-mon[30018]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:20.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:20 smithi090 ceph-mon[30018]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:20.428 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:24:20.428 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705", 2024-09-16T08:24:20.429 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-09-16T08:24:20.429 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:24:20.429 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-09-16T08:24:20.429 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "", 2024-09-16T08:24:20.429 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 image", 2024-09-16T08:24:20.429 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-09-16T08:24:20.429 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:24:20.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:20 smithi193 ceph-mon[25614]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:20.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:20 smithi193 ceph-mon[25614]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:20.778 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 Cephadm Agent(s) are not reporting. Hosts may be offline 2024-09-16T08:24:20.778 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] CEPHADM_AGENT_DOWN: 1 Cephadm Agent(s) are not reporting. Hosts may be offline 2024-09-16T08:24:20.778 INFO:teuthology.orchestra.run.smithi090.stdout: Cephadm agent on host smithi193 has not reported in 60.0 seconds. Agent is assumed down and host may be offline. 2024-09-16T08:24:21.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[30018]: from='client.25004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:21.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[30018]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:21.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3678609130' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:24:21.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[30018]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:21.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1891057016' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:24:21.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[23701]: from='client.25004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:21.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[23701]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:21.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3678609130' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:24:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[23701]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:21 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1891057016' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:24:21.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:21 smithi193 ceph-mon[25614]: from='client.25004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:21.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:21 smithi193 ceph-mon[25614]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:21.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:21 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3678609130' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:24:21.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:21 smithi193 ceph-mon[25614]: from='client.15147 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:21.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:21 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1891057016' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:24:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:23 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:23 smithi090 ceph-mon[30018]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:23.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:23 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:23.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:23 smithi090 ceph-mon[23701]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:23.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:23 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:23.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:23 smithi193 ceph-mon[25614]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:25.311 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:25 smithi090 ceph-mon[30018]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:25.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:25 smithi090 ceph-mon[23701]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:25.389 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:25 smithi193 ceph-mon[25614]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:27.311 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:27 smithi090 ceph-mon[30018]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:27.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:27 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:27 smithi090 ceph-mon[23701]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:27.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:27 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:27.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:27 smithi193 ceph-mon[25614]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T08:24:27.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:27 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:29.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:29 smithi090 ceph-mon[30018]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:24:29.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:29 smithi090 ceph-mon[23701]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:24:29.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:29 smithi193 ceph-mon[25614]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T08:24:30.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:24:29 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:24:29] "GET /metrics HTTP/1.1" 200 38505 "" "Prometheus/2.43.0" 2024-09-16T08:24:31.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:31 smithi193 ceph-mon[25614]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:31.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:31 smithi090 ceph-mon[23701]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:31.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:31 smithi090 ceph-mon[30018]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:33.345 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:33 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:33.345 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:33 smithi193 ceph-mon[25614]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:33.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:33 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:33.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:33 smithi090 ceph-mon[23701]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:33 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:33.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:33 smithi090 ceph-mon[30018]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:35.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:35 smithi193 ceph-mon[25614]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:35.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:35 smithi090 ceph-mon[30018]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:35.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:35 smithi090 ceph-mon[23701]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:37.423 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:37 smithi090 ceph-mon[23701]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:37.424 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:37 smithi090 ceph-mon[30018]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:37.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:37 smithi193 ceph-mon[25614]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:39 smithi090 ceph-mon[23701]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:39 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:39 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:39 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:39 smithi090 ceph-mon[30018]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:39 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:39 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:39 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:39 smithi193 ceph-mon[25614]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:39 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:39 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:39 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:40.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:24:39 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:24:39] "GET /metrics HTTP/1.1" 200 38505 "" "Prometheus/2.43.0" 2024-09-16T08:24:41.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:41 smithi193 ceph-mon[25614]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:41.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:41 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:41.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:41 smithi090 ceph-mon[23701]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:41.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:41 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:41.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:41 smithi090 ceph-mon[30018]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:41.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:41 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:43.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:43 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:43.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:43 smithi193 ceph-mon[25614]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:43.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:43 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:43.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:43 smithi090 ceph-mon[30018]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:43.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:43 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:43.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:43 smithi090 ceph-mon[23701]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:45.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:45 smithi193 ceph-mon[25614]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:45.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:45 smithi090 ceph-mon[30018]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:45.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:45 smithi090 ceph-mon[23701]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:47.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:47 smithi193 ceph-mon[25614]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:47.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:47 smithi090 ceph-mon[30018]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:47.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:47 smithi090 ceph-mon[23701]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:49.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:49 smithi193 ceph-mon[25614]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:49.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:49 smithi090 ceph-mon[30018]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:49.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:49 smithi090 ceph-mon[23701]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:50.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:24:49 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:24:49] "GET /metrics HTTP/1.1" 200 38505 "" "Prometheus/2.43.0" 2024-09-16T08:24:51.078 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-09-16T08:24:51.669 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:24:51.669 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi090 smithi090 running 13s ago 9m - - 2024-09-16T08:24:51.669 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi193 smithi193 error 33s ago 9m - - 2024-09-16T08:24:51.669 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.a smithi090 *:9093,9094 running (3m) 13s ago 4m 27.4M - 0.25.0 c8568f914cd2 3c66ea902fdc 2024-09-16T08:24:51.669 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.a smithi193 *:3000 running (4m) 33s ago 4m 87.6M - 9.4.7 954c08fa6188 d2a25f536ef6 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:iscsi.foo.smithi090.jxkvvk smithi090 *:5000 running (2m) 13s ago 2m 53.2M - 3.8 fd1d6ca2b4a5 f7eb9aafebf0 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi193 *:8443,8765 running (8m) 33s ago 8m 439M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 4b116b0d4a11 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi090 *:9283,8765 running (10m) 13s ago 10m 502M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 92b236ae51c0 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (10m) 13s ago 10m 53.0M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 87c2b7a329d6 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi193 running (9m) 33s ago 9m 41.4M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a45dcb4d7793 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi090 running (9m) 13s ago 9m 40.8M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 fddfc4718a9c 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.a smithi090 *:9100 running (4m) 13s ago 4m 16.9M - 1.5.0 0da6a335fe13 4a6ade533b17 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.b smithi193 *:9100 running (4m) 33s ago 4m 16.0M - 1.5.0 0da6a335fe13 46a72e78b0a2 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (8m) 13s ago 8m 51.7M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 d8ac704cb999 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (7m) 13s ago 7m 58.2M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0703646c72eb 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (7m) 13s ago 7m 47.7M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3f0553bc3456 2024-09-16T08:24:51.670 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (7m) 13s ago 7m 58.4M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90097820090a 2024-09-16T08:24:51.671 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi193 running (6m) 33s ago 6m 51.4M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 389bfe865f47 2024-09-16T08:24:51.671 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi193 running (6m) 33s ago 6m 49.1M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 b5fa3f6ab4bf 2024-09-16T08:24:51.671 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi193 running (5m) 33s ago 5m 59.7M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 dc9317ffcb0d 2024-09-16T08:24:51.671 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi193 running (5m) 33s ago 5m 59.8M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 9a7223ca10b1 2024-09-16T08:24:51.671 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.a smithi193 *:9095 running (3m) 33s ago 5m 49.7M - 2.43.0 a07b618ecd1d 0416160d55ca 2024-09-16T08:24:51.671 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi090.fbrjws smithi090 *:8000 running (3m) 13s ago 3m 76.3M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 6bd392324db2 2024-09-16T08:24:51.671 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi193.gjyvkw smithi193 *:8000 running (3m) 33s ago 3m 75.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 351896279a6f 2024-09-16T08:24:51.671 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi090.ebelfk smithi090 *:80 running (2m) 13s ago 2m 75.7M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 374e9c0e942b 2024-09-16T08:24:51.671 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi193.cgoeeh smithi193 *:80 running (2m) 33s ago 2m 76.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cf9d24a65b38 2024-09-16T08:24:51.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:51 smithi193 ceph-mon[25614]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:51.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:51 smithi090 ceph-mon[30018]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:51.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:51 smithi090 ceph-mon[23701]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:52.032 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:24:52.032 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-09-16T08:24:52.032 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 3 2024-09-16T08:24:52.032 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:24:52.032 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-09-16T08:24:52.032 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:24:52.032 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:24:52.032 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-09-16T08:24:52.033 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 8 2024-09-16T08:24:52.033 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:24:52.033 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw": { 2024-09-16T08:24:52.033 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:24:52.033 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:24:52.033 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-09-16T08:24:52.033 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 17 2024-09-16T08:24:52.033 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:24:52.033 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:24:52.332 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:24:52.332 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705", 2024-09-16T08:24:52.332 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-09-16T08:24:52.332 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:24:52.333 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-09-16T08:24:52.333 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "", 2024-09-16T08:24:52.333 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 image", 2024-09-16T08:24:52.333 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-09-16T08:24:52.333 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:24:52.686 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:52 smithi090 ceph-mon[30018]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:52.686 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:52 smithi090 ceph-mon[30018]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:52.686 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:52 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/542043181' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:24:52.687 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 Cephadm Agent(s) are not reporting. Hosts may be offline 2024-09-16T08:24:52.687 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] CEPHADM_AGENT_DOWN: 1 Cephadm Agent(s) are not reporting. Hosts may be offline 2024-09-16T08:24:52.687 INFO:teuthology.orchestra.run.smithi090.stdout: Cephadm agent on host smithi193 has not reported in 60.0 seconds. Agent is assumed down and host may be offline. 2024-09-16T08:24:52.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:52 smithi090 ceph-mon[23701]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:52.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:52 smithi090 ceph-mon[23701]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:52.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:52 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/542043181' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:24:52.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:52 smithi193 ceph-mon[25614]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:52.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:52 smithi193 ceph-mon[25614]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:52.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:52 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/542043181' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:24:53.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:53 smithi193 ceph-mon[25614]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:53.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:53 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:53.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:53 smithi193 ceph-mon[25614]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:53.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:53 smithi193 ceph-mon[25614]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:53.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:53 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2202938015' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:24:53.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[30018]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:53.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:53.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[30018]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:53.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[30018]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:53.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2202938015' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:24:53.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[23701]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:53.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:24:53.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[23701]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:53.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[23701]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:24:53.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:53 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2202938015' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:24:55.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:55 smithi193 ceph-mon[25614]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:55.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:55 smithi090 ceph-mon[30018]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:55.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:55 smithi090 ceph-mon[23701]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:56 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:56.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:56 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:56.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:56 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:24:57.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:57 smithi193 ceph-mon[25614]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:57.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:57 smithi090 ceph-mon[30018]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:57.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:57 smithi090 ceph-mon[23701]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:24:59.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:59 smithi193 ceph-mon[25614]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:59.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:59 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:59.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:59 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:59.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:24:59 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:59.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:59 smithi090 ceph-mon[30018]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:59.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:59 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:59.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:59 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:59.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:24:59 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:59.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:59 smithi090 ceph-mon[23701]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:24:59.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:59 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:59.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:59 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:24:59.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:24:59 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:00.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:24:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:24:59] "GET /metrics HTTP/1.1" 200 38503 "" "Prometheus/2.43.0" 2024-09-16T08:25:01.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:01 smithi193 ceph-mon[25614]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:01.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:01.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:01.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:01 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:01.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:01 smithi090 ceph-mon[30018]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:01.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:01.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:01.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:01 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:01.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:01 smithi090 ceph-mon[23701]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:01.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:01.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:01.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:01 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:02.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:02 smithi090 ceph-mon[23701]: Upgrade: Target is version 19.3.0-4917-g8293d73f (unknown) 2024-09-16T08:25:02.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:02 smithi090 ceph-mon[23701]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce'] 2024-09-16T08:25:02.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:02 smithi090 ceph-mon[23701]: Upgrade: Need to upgrade myself (mgr.y) 2024-09-16T08:25:02.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:02 smithi090 ceph-mon[23701]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce on smithi193 2024-09-16T08:25:02.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:02 smithi090 ceph-mon[30018]: Upgrade: Target is version 19.3.0-4917-g8293d73f (unknown) 2024-09-16T08:25:02.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:02 smithi090 ceph-mon[30018]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce'] 2024-09-16T08:25:02.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:02 smithi090 ceph-mon[30018]: Upgrade: Need to upgrade myself (mgr.y) 2024-09-16T08:25:02.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:02 smithi090 ceph-mon[30018]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce on smithi193 2024-09-16T08:25:03.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:02 smithi193 ceph-mon[25614]: Upgrade: Target is version 19.3.0-4917-g8293d73f (unknown) 2024-09-16T08:25:03.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:02 smithi193 ceph-mon[25614]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce'] 2024-09-16T08:25:03.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:02 smithi193 ceph-mon[25614]: Upgrade: Need to upgrade myself (mgr.y) 2024-09-16T08:25:03.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:02 smithi193 ceph-mon[25614]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce on smithi193 2024-09-16T08:25:03.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:03 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:03.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:03 smithi090 ceph-mon[30018]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:03.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:03 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:03.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:03 smithi090 ceph-mon[23701]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:04.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:03 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:04.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:03 smithi193 ceph-mon[25614]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:05.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:05 smithi090 ceph-mon[30018]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:05.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:05 smithi090 ceph-mon[23701]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:05.852 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:05 smithi193 ceph-mon[25614]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:07.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:07 smithi090 ceph-mon[30018]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:07.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:07 smithi090 ceph-mon[23701]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:08.001 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:07 smithi193 ceph-mon[25614]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:09.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:09 smithi090 ceph-mon[23701]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:09.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:09 smithi090 ceph-mon[30018]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:10.001 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:09 smithi193 ceph-mon[25614]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:10.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:25:09] "GET /metrics HTTP/1.1" 200 38503 "" "Prometheus/2.43.0" 2024-09-16T08:25:11.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:11 smithi090 ceph-mon[23701]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:11.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:11 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:11.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:11 smithi090 ceph-mon[30018]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:11.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:11 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:12.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:11 smithi193 ceph-mon[25614]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:12.160 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:11 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:13 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:13.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:13 smithi193 ceph-mon[25614]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:13.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:13 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:13.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:13 smithi090 ceph-mon[23701]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:13 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:13 smithi090 ceph-mon[30018]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:15.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:15 smithi193 ceph-mon[25614]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:15.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:15 smithi090 ceph-mon[30018]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:15.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:15 smithi090 ceph-mon[23701]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:17.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:17 smithi090 ceph-mon[30018]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:17.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:17 smithi090 ceph-mon[23701]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:18.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:17 smithi193 ceph-mon[25614]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:19.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:19 smithi090 ceph-mon[30018]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:19.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:19.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:19.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:19 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:19.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:19 smithi090 ceph-mon[23701]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:19.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:19.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:19.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:19 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:20.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:19 smithi193 ceph-mon[25614]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:20.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:20.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:20.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:19 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:20.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:19 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:25:19] "GET /metrics HTTP/1.1" 200 38504 "" "Prometheus/2.43.0" 2024-09-16T08:25:21.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:21 smithi090 ceph-mon[30018]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:21.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:21 smithi090 ceph-mon[23701]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:22.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:21 smithi193 ceph-mon[25614]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:22.987 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-09-16T08:25:23.558 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:25:23.558 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi090 smithi090 running 4s ago 10m - - 2024-09-16T08:25:23.558 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi193 smithi193 error 65s ago 9m - - 2024-09-16T08:25:23.558 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.a smithi090 *:9093,9094 running (4m) 4s ago 5m 27.4M - 0.25.0 c8568f914cd2 3c66ea902fdc 2024-09-16T08:25:23.558 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.a smithi193 *:3000 running (5m) 65s ago 5m 87.6M - 9.4.7 954c08fa6188 d2a25f536ef6 2024-09-16T08:25:23.558 INFO:teuthology.orchestra.run.smithi090.stdout:iscsi.foo.smithi090.jxkvvk smithi090 *:5000 running (3m) 4s ago 3m 53.4M - 3.8 fd1d6ca2b4a5 f7eb9aafebf0 2024-09-16T08:25:23.558 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi193 *:8443,8765 running (9m) 65s ago 9m 439M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 4b116b0d4a11 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi090 *:9283,8765 running (11m) 4s ago 10m 503M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 92b236ae51c0 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (11m) 4s ago 11m 51.5M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 87c2b7a329d6 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi193 running (9m) 65s ago 9m 41.4M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a45dcb4d7793 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi090 running (9m) 4s ago 9m 41.0M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 fddfc4718a9c 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.a smithi090 *:9100 running (4m) 4s ago 4m 17.1M - 1.5.0 0da6a335fe13 4a6ade533b17 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.b smithi193 *:9100 running (4m) 65s ago 4m 16.0M - 1.5.0 0da6a335fe13 46a72e78b0a2 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (8m) 4s ago 8m 52.2M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 d8ac704cb999 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (8m) 4s ago 8m 58.4M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0703646c72eb 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (8m) 4s ago 8m 48.0M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3f0553bc3456 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (7m) 4s ago 7m 58.7M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90097820090a 2024-09-16T08:25:23.559 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi193 running (7m) 65s ago 7m 51.4M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 389bfe865f47 2024-09-16T08:25:23.560 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi193 running (6m) 65s ago 6m 49.1M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 b5fa3f6ab4bf 2024-09-16T08:25:23.560 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi193 running (6m) 65s ago 6m 59.7M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 dc9317ffcb0d 2024-09-16T08:25:23.560 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi193 running (6m) 65s ago 6m 59.8M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 9a7223ca10b1 2024-09-16T08:25:23.560 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.a smithi193 *:9095 running (4m) 65s ago 5m 49.7M - 2.43.0 a07b618ecd1d 0416160d55ca 2024-09-16T08:25:23.560 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi090.fbrjws smithi090 *:8000 running (3m) 4s ago 3m 76.7M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 6bd392324db2 2024-09-16T08:25:23.560 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi193.gjyvkw smithi193 *:8000 running (3m) 65s ago 3m 75.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 351896279a6f 2024-09-16T08:25:23.560 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi090.ebelfk smithi090 *:80 running (3m) 4s ago 3m 76.1M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 374e9c0e942b 2024-09-16T08:25:23.560 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi193.cgoeeh smithi193 *:80 running (3m) 65s ago 3m 76.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cf9d24a65b38 2024-09-16T08:25:23.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:23 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:23.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:23 smithi090 ceph-mon[30018]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:23.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:23 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:23.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:23 smithi090 ceph-mon[23701]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:23.888 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:25:23.888 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-09-16T08:25:23.888 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 3 2024-09-16T08:25:23.888 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:25:23.888 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-09-16T08:25:23.888 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:25:23.888 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 8 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw": { 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 17 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:25:23.889 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:25:24.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:23 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:24.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:23 smithi193 ceph-mon[25614]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:24.168 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:25:24.168 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T08:25:24.168 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-09-16T08:25:24.168 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:25:24.168 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-09-16T08:25:24.168 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "0/23 daemons upgraded", 2024-09-16T08:25:24.168 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce image on host smithi193", 2024-09-16T08:25:24.168 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-09-16T08:25:24.168 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:25:24.494 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 Cephadm Agent(s) are not reporting. Hosts may be offline 2024-09-16T08:25:24.494 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] CEPHADM_AGENT_DOWN: 1 Cephadm Agent(s) are not reporting. Hosts may be offline 2024-09-16T08:25:24.494 INFO:teuthology.orchestra.run.smithi090.stdout: Cephadm agent on host smithi193 has not reported in 60.0 seconds. Agent is assumed down and host may be offline. 2024-09-16T08:25:24.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:24 smithi090 ceph-mon[23701]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:24.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:24 smithi090 ceph-mon[23701]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:24.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:24 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/759923002' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:25:24.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:24 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3332950107' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:25:24.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:24 smithi090 ceph-mon[30018]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:24.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:24 smithi090 ceph-mon[30018]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:24.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:24 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/759923002' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:25:24.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:24 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3332950107' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:25:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:24 smithi193 ceph-mon[25614]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:24 smithi193 ceph-mon[25614]: from='client.25070 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:24 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/759923002' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:25:25.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:24 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3332950107' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:25:25.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:25 smithi090 ceph-mon[23701]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:25.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:25 smithi090 ceph-mon[23701]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:25.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:25 smithi090 ceph-mon[23701]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:25.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:25 smithi090 ceph-mon[30018]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:25.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:25 smithi090 ceph-mon[30018]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:25.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:25 smithi090 ceph-mon[30018]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:26.001 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:25 smithi193 ceph-mon[25614]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:26.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:25 smithi193 ceph-mon[25614]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:26.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:25 smithi193 ceph-mon[25614]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:26.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:26 smithi090 ceph-mon[23701]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:26.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:26 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:26.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:26 smithi090 ceph-mon[30018]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:26.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:26 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:26.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:26 smithi193 ceph-mon[25614]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:26.895 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:26 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:29.468 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:29 smithi193 ceph-mon[25614]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:29.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:29 smithi090 ceph-mon[30018]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:29.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:29 smithi090 ceph-mon[23701]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:30.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:29 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:25:29] "GET /metrics HTTP/1.1" 200 38506 "" "Prometheus/2.43.0" 2024-09-16T08:25:31.164 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:31 smithi193 ceph-mon[25614]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:31.164 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:31 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:31.164 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:31 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:25:31.164 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:31 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:25:31.165 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:31 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:25:31.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[23701]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:31.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:31.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:25:31.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:25:31.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:25:31.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[30018]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:31.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:31.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:25:31.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:25:31.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:31 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:25:32.130 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:32 smithi090 ceph-mon[30018]: Upgrade: Updating mgr.x 2024-09-16T08:25:32.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:32 smithi090 ceph-mon[23701]: Upgrade: Updating mgr.x 2024-09-16T08:25:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:32 smithi090 ceph-mon[23701]: Deploying daemon mgr.x on smithi193 2024-09-16T08:25:32.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:32 smithi193 systemd[1]: Stopping Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:25:32.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:32 smithi193 ceph-mon[25614]: Upgrade: Updating mgr.x 2024-09-16T08:25:32.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:32 smithi193 ceph-mon[25614]: Deploying daemon mgr.x on smithi193 2024-09-16T08:25:32.501 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:32 smithi090 ceph-mon[30018]: Deploying daemon mgr.x on smithi193 2024-09-16T08:25:32.718 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:32 smithi193 podman[68019]: 2024-09-16 08:25:32.439754677 +0000 UTC m=+0.138077065 container died 4b116b0d4a11e5b7b688116b7e1912ade63c7ced52da0be3902a86d87281fbf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2) 2024-09-16T08:25:33.002 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:32 smithi193 podman[68019]: 2024-09-16 08:25:32.854396269 +0000 UTC m=+0.552718668 container cleanup 4b116b0d4a11e5b7b688116b7e1912ade63c7ced52da0be3902a86d87281fbf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-16T08:25:33.002 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:32 smithi193 bash[68019]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x 2024-09-16T08:25:33.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:33 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:33.415 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:33 smithi193 ceph-mon[25614]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:33.415 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:33 smithi193 podman[68030]: 2024-09-16 08:25:33.169897926 +0000 UTC m=+0.730213219 container remove 4b116b0d4a11e5b7b688116b7e1912ade63c7ced52da0be3902a86d87281fbf1 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T08:25:33.415 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:33 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-16T08:25:33.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:33 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:33.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:33 smithi090 ceph-mon[23701]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:33.510 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:33 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:33.510 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:33 smithi090 ceph-mon[30018]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:33.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:33 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-16T08:25:33.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:33 smithi193 systemd[1]: Stopped Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:25:33.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:33 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Consumed 11.191s CPU time. 2024-09-16T08:25:33.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:33 smithi193 systemd[1]: Starting Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:25:33.812 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:25:33 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:25:33.509Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi193.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.193:8443: connect: connection refused" 2024-09-16T08:25:34.235 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:33 smithi193 podman[68160]: 2024-09-16 08:25:33.759553441 +0000 UTC m=+0.015905373 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T08:25:34.235 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:34 smithi193 podman[68160]: 2024-09-16 08:25:34.005707589 +0000 UTC m=+0.262059521 container create 23e7ac0df0bffa3e17b0cd1d47c57447e79249b6d6364cef441890f1d435696e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:25:35.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:35 smithi193 ceph-mon[25614]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:35.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:34 smithi193 podman[68160]: 2024-09-16 08:25:34.978776117 +0000 UTC m=+1.235128048 container init 23e7ac0df0bffa3e17b0cd1d47c57447e79249b6d6364cef441890f1d435696e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:25:35.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:34 smithi193 podman[68160]: 2024-09-16 08:25:34.982119248 +0000 UTC m=+1.238471187 container start 23e7ac0df0bffa3e17b0cd1d47c57447e79249b6d6364cef441890f1d435696e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:25:35.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:35 smithi193 ceph-mgr[68211]: -- 172.21.15.193:0/3579464889 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cb8b9390e0 con 0x55cb8b917400 2024-09-16T08:25:35.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:35 smithi193 bash[68160]: 23e7ac0df0bffa3e17b0cd1d47c57447e79249b6d6364cef441890f1d435696e 2024-09-16T08:25:35.253 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:35 smithi193 systemd[1]: Started Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:25:35.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:35 smithi090 ceph-mon[30018]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:35.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:35 smithi090 ceph-mon[23701]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:36.479 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:36 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.479 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:36 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.479 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:36 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:36.479 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:36 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.479 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:36 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.479 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:36 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.480 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:36 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:36.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:36.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:36 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:37.464 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:37 smithi090 ceph-mon[30018]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:37.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:37 smithi090 ceph-mon[23701]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:37.474 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:37 smithi193 ceph-mon[25614]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:39.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:39 smithi193 ceph-mon[25614]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:39.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:39 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:39.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:39 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:39.424 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:39 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:39 smithi090 ceph-mon[23701]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:39 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:39 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:39.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:39 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:39.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:39 smithi090 ceph-mon[30018]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:39 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:39 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:39.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:39 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:39.957 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:39 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:25:39] "GET /metrics HTTP/1.1" 200 38506 "" "Prometheus/2.43.0" 2024-09-16T08:25:40.639 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:40 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:25:40] ENGINE Bus STARTING 2024-09-16T08:25:40.639 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:40 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: CherryPy Checker: 2024-09-16T08:25:40.639 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:40 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: The Application mounted at '' has an empty config. 2024-09-16T08:25:40.640 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:40 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: 2024-09-16T08:25:40.640 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:40 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:25:40] ENGINE Serving on http://:::9283 2024-09-16T08:25:40.640 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:40 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:25:40] ENGINE Bus STARTED 2024-09-16T08:25:41.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[30018]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:41.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[30018]: Standby manager daemon x restarted 2024-09-16T08:25:41.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[30018]: Standby manager daemon x started 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[23701]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[23701]: Standby manager daemon x restarted 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[23701]: Standby manager daemon x started 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:25:41.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:25:41.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:25:41.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:25:41.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:41.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:41 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:41 smithi193 ceph-mon[25614]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:41 smithi193 ceph-mon[25614]: Standby manager daemon x restarted 2024-09-16T08:25:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:41 smithi193 ceph-mon[25614]: Standby manager daemon x started 2024-09-16T08:25:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:41 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:25:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:41 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:25:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:41 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:25:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:41 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:25:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:41 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:41.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:41 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:42.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:42 smithi193 ceph-mon[25614]: mgrmap e22: y(active, since 5m), standbys: x 2024-09-16T08:25:42.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:42.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:42.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:42 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:42.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:42 smithi090 ceph-mon[30018]: mgrmap e22: y(active, since 5m), standbys: x 2024-09-16T08:25:42.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:42.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:42.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:42 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:42.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:42 smithi090 ceph-mon[23701]: mgrmap e22: y(active, since 5m), standbys: x 2024-09-16T08:25:42.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:42.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:42.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:42 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:43.062 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:25:42 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:25:42.810Z caller=notify.go:743 level=info component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify success" attempts=7 2024-09-16T08:25:43.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:43 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:43.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:43 smithi193 ceph-mon[25614]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:43 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:43 smithi090 ceph-mon[30018]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:43.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:43 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:43.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:43 smithi090 ceph-mon[23701]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:45.235 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:45 smithi193 ceph-mon[25614]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:45.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:45 smithi090 ceph-mon[30018]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:45.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:45 smithi090 ceph-mon[23701]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:47.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:47 smithi193 ceph-mon[25614]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:47.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:47 smithi090 ceph-mon[23701]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:47.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:47 smithi090 ceph-mon[30018]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:49.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:49 smithi193 ceph-mon[25614]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:49.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:49 smithi090 ceph-mon[30018]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:49.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:49 smithi090 ceph-mon[23701]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:25:50.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:49 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:25:49] "GET /metrics HTTP/1.1" 200 38506 "" "Prometheus/2.43.0" 2024-09-16T08:25:51.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:51 smithi193 ceph-mon[25614]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:51.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:51 smithi090 ceph-mon[30018]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:51.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:51 smithi090 ceph-mon[23701]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:53.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:53 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:53.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:53 smithi193 ceph-mon[25614]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:53.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:53 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:53.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:53 smithi090 ceph-mon[23701]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:53.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:53 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:25:53.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:53 smithi090 ceph-mon[30018]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T08:25:53.930 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:25:53] ENGINE Bus STOPPING 2024-09-16T08:25:53.930 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:25:53] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:25:53.931 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:25:53] ENGINE Bus STOPPED 2024-09-16T08:25:54.188 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:53 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ignoring --setuser ceph since I am not root 2024-09-16T08:25:54.188 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:53 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ignoring --setgroup ceph since I am not root 2024-09-16T08:25:54.188 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:53 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:53.987+0000 7f21b08cc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T08:25:54.188 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:54.043+0000 7f21b08cc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T08:25:54.189 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:54.187+0000 7f21b08cc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T08:25:54.539 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:54.253+0000 7f21b08cc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T08:25:54.539 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:54.398+0000 7f21b08cc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:25:54.813 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:54.538+0000 7f21b08cc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T08:25:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:25:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:25:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: Activating manager daemon x 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: osdmap e96: 8 total, 8 up, 8 in 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: mgrmap e23: x(active, starting, since 0.0084582s) 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:25:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[30018]: Manager daemon x is now available 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:54.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: Activating manager daemon x 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: osdmap e96: 8 total, 8 up, 8 in 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: mgrmap e23: x(active, starting, since 0.0084582s) 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:25:54.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:25:54.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-mon[23701]: Manager daemon x is now available 2024-09-16T08:25:54.861 INFO:teuthology.orchestra.run.smithi090.stderr:Error ENOTSUP: Warning: due to ceph-mgr restart, some PG states may not be up to date 2024-09-16T08:25:54.862 INFO:teuthology.orchestra.run.smithi090.stderr:Module 'orchestrator' is not enabled/loaded (required by command 'orch upgrade status'): use `ceph mgr module enable orchestrator` to enable it 2024-09-16T08:25:55.002 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: 2024-09-16T08:25:54.854+0000 7f86e8d8e640 -1 mgr.server reply reply (95) Operation not supported Warning: due to ceph-mgr restart, some PG states may not be up to date 2024-09-16T08:25:55.002 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Module 'orchestrator' is not enabled/loaded (required by command 'orch upgrade status'): use `ceph mgr module enable orchestrator` to enable it 2024-09-16T08:25:55.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:55.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:55.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd=[{"prefix": "mgr fail", "who": "y"}]: dispatch 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: Activating manager daemon x 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: osdmap e96: 8 total, 8 up, 8 in 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.14457 172.21.15.90:0/819243596' entity='mgr.y' cmd='[{"prefix": "mgr fail", "who": "y"}]': finished 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: mgrmap e23: x(active, starting, since 0.0084582s) 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:25:55.003 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:25:55.004 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:54 smithi193 ceph-mon[25614]: Manager daemon x is now available 2024-09-16T08:25:55.126 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:54.862+0000 7f21b08cc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T08:25:55.127 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:54 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:54.933+0000 7f21b08cc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T08:25:55.127 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:55.125+0000 7f21b08cc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T08:25:55.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:25:55] ENGINE Bus STARTING 2024-09-16T08:25:55.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:25:55] ENGINE Serving on http://:::9283 2024-09-16T08:25:55.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:25:55] ENGINE Bus STARTED 2024-09-16T08:25:55.558 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch ps' 2024-09-16T08:25:55.783 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:25:56.137 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:56.072+0000 7f21b08cc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T08:25:56.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: mgrmap e24: x(active, since 1.01198s) 2024-09-16T08:25:56.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='client.25115 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:56.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:25:56.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:25:56.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: mgrmap e24: x(active, since 1.01198s) 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='client.25115 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.140 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T08:25:56.141 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:55 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T08:25:56.156 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: mgrmap e24: x(active, since 1.01198s) 2024-09-16T08:25:56.156 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='client.25115 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:56.157 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:25:56.157 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.157 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.157 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.157 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.157 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.158 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.158 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:56.158 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:25:56.158 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:25:56.158 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:25:56.158 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:25:56.159 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:25:56.159 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:25:56.159 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:25:56.159 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:25:56.159 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:25:56.159 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T08:25:56.159 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:55 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T08:25:56.500 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: 2024-09-16T08:25:56.332+0000 7f86adadb640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:08:25:56] ENGINE Error in HTTPServer.serve 2024-09-16T08:25:56.500 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Traceback (most recent call last): 2024-09-16T08:25:56.500 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: self._connections.run(self.expiration_interval) 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: self._run(expiration_interval) 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: s = self.context.wrap_socket( 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return self.sslsocket_class._create( 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:25:56.501 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: self.do_handshake() 2024-09-16T08:25:56.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:25:56.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: self._sslobj.do_handshake() 2024-09-16T08:25:56.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:25:56.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: 2024-09-16T08:25:56.562 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:56.143+0000 7f21b08cc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T08:25:56.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: Migrating certs/keys for iscsi.foo spec to cert store 2024-09-16T08:25:56.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: Migrating certs/keys for rgw.foo spec to cert store 2024-09-16T08:25:56.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: Migrating certs/keys for rgw.smpl spec to cert store 2024-09-16T08:25:56.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: Checking for cert/key for grafana.a 2024-09-16T08:25:56.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: Migrating grafana.a cert to cert store 2024-09-16T08:25:56.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: Migrating grafana.a key to cert store 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[23701]: Reconfiguring daemon agent.smithi090 on smithi090 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: Migrating certs/keys for iscsi.foo spec to cert store 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: Migrating certs/keys for rgw.foo spec to cert store 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: Migrating certs/keys for rgw.smpl spec to cert store 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: Checking for cert/key for grafana.a 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: Migrating grafana.a cert to cert store 2024-09-16T08:25:56.874 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: Migrating grafana.a key to cert store 2024-09-16T08:25:56.875 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:25:56.875 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-16T08:25:56.875 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:25:56.875 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:25:56.875 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-mon[30018]: Reconfiguring daemon agent.smithi090 on smithi090 2024-09-16T08:25:57.135 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:56.872+0000 7f21b08cc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T08:25:57.135 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:56 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:56.987+0000 7f21b08cc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T08:25:57.179 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: Migrating certs/keys for iscsi.foo spec to cert store 2024-09-16T08:25:57.179 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: Migrating certs/keys for rgw.foo spec to cert store 2024-09-16T08:25:57.179 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: Migrating certs/keys for rgw.smpl spec to cert store 2024-09-16T08:25:57.179 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: Checking for cert/key for grafana.a 2024-09-16T08:25:57.180 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: Migrating grafana.a cert to cert store 2024-09-16T08:25:57.180 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: Migrating grafana.a key to cert store 2024-09-16T08:25:57.180 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:25:57.180 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-16T08:25:57.180 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:25:57.180 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:25:57.181 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:56 smithi193 ceph-mon[25614]: Reconfiguring daemon agent.smithi090 on smithi090 2024-09-16T08:25:57.444 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:57.134+0000 7f21b08cc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:25:57.444 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:57.203+0000 7f21b08cc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T08:25:57.444 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:57.365+0000 7f21b08cc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T08:25:57.444 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:57.443+0000 7f21b08cc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T08:25:57.768 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:57.522+0000 7f21b08cc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T08:25:57.768 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:57.690+0000 7f21b08cc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi090 smithi090 starting - - - - 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi193 smithi193 running 16s ago 10m - - 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.a smithi090 *:9093,9094 running (4m) 19s ago 5m 27.4M - 0.25.0 c8568f914cd2 3c66ea902fdc 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.a smithi193 *:3000 running (5m) 16s ago 5m 86.0M - 9.4.7 954c08fa6188 d2a25f536ef6 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:iscsi.foo.smithi090.jxkvvk smithi090 *:5000 running (3m) 19s ago 3m 53.4M - 3.8 fd1d6ca2b4a5 f7eb9aafebf0 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi193 *:8443,9283,8765 running (24s) 16s ago 9m 124M - 19.3.0-4917-g8293d73f 425e316ba8eb 23e7ac0df0bf 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi090 *:9283,8765 running (11m) 19s ago 11m 503M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 92b236ae51c0 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (11m) 19s ago 11m 52.0M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 87c2b7a329d6 2024-09-16T08:25:57.921 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi193 running (10m) 16s ago 10m 43.3M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a45dcb4d7793 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi090 running (10m) 19s ago 10m 41.7M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 fddfc4718a9c 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.a smithi090 *:9100 running (5m) 19s ago 5m 16.6M - 1.5.0 0da6a335fe13 4a6ade533b17 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.b smithi193 *:9100 running (5m) 16s ago 5m 16.4M - 1.5.0 0da6a335fe13 46a72e78b0a2 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (9m) 19s ago 9m 52.3M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 d8ac704cb999 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (9m) 19s ago 9m 58.6M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0703646c72eb 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (8m) 19s ago 8m 48.1M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3f0553bc3456 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (8m) 19s ago 8m 58.9M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90097820090a 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi193 running (7m) 16s ago 7m 52.1M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 389bfe865f47 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi193 running (7m) 16s ago 7m 49.8M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 b5fa3f6ab4bf 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi193 running (7m) 16s ago 7m 60.2M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 dc9317ffcb0d 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi193 running (6m) 16s ago 6m 60.4M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 9a7223ca10b1 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.a smithi193 *:9095 running (4m) 16s ago 6m 52.1M - 2.43.0 a07b618ecd1d 0416160d55ca 2024-09-16T08:25:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi090.fbrjws smithi090 *:8000 running (4m) 19s ago 4m 76.9M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 6bd392324db2 2024-09-16T08:25:57.923 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi193.gjyvkw smithi193 *:8000 running (4m) 16s ago 4m 76.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 351896279a6f 2024-09-16T08:25:57.923 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi090.ebelfk smithi090 *:80 running (3m) 19s ago 3m 76.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 374e9c0e942b 2024-09-16T08:25:57.923 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi193.cgoeeh smithi193 *:80 running (3m) 16s ago 3m 77.0M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cf9d24a65b38 2024-09-16T08:25:58.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[23701]: Reconfiguring daemon agent.smithi193 on smithi193 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[23701]: [16/Sep/2024:08:25:56] ENGINE Bus STARTING 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[23701]: Deploying cephadm binary to smithi090 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[23701]: [16/Sep/2024:08:25:56] ENGINE Serving on http://172.21.15.193:8765 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[23701]: [16/Sep/2024:08:25:56] ENGINE Serving on https://172.21.15.193:7150 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[23701]: [16/Sep/2024:08:25:56] ENGINE Bus STARTED 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[23701]: [16/Sep/2024:08:25:56] ENGINE Error in HTTPServer.serve 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._run(expiration_interval) 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:25:58.022 INFO:journalctl@ceph.mon.a.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: s = self.context.wrap_socket( 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.sslsocket_class._create( 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: self.do_handshake() 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._sslobj.do_handshake() 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:25:58.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[23701]: mgrmap e25: x(active, since 3s) 2024-09-16T08:25:58.024 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:57.767+0000 7f21b08cc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T08:25:58.024 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:57.848+0000 7f21b08cc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T08:25:58.024 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:57.961+0000 7f21b08cc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T08:25:58.024 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:58.020+0000 7f21b08cc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T08:25:58.024 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[30018]: Reconfiguring daemon agent.smithi193 on smithi193 2024-09-16T08:25:58.024 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[30018]: [16/Sep/2024:08:25:56] ENGINE Bus STARTING 2024-09-16T08:25:58.024 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[30018]: Deploying cephadm binary to smithi090 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[30018]: [16/Sep/2024:08:25:56] ENGINE Serving on http://172.21.15.193:8765 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[30018]: [16/Sep/2024:08:25:56] ENGINE Serving on https://172.21.15.193:7150 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[30018]: [16/Sep/2024:08:25:56] ENGINE Bus STARTED 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[30018]: [16/Sep/2024:08:25:56] ENGINE Error in HTTPServer.serve 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout: self._run(expiration_interval) 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:25:58.025 INFO:journalctl@ceph.mon.c.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout: s = self.context.wrap_socket( 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout: return self.sslsocket_class._create( 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout: self.do_handshake() 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout: self._sslobj.do_handshake() 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:25:58.026 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:57 smithi090 ceph-mon[30018]: mgrmap e25: x(active, since 3s) 2024-09-16T08:25:58.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:57 smithi193 ceph-mon[25614]: Reconfiguring daemon agent.smithi193 on smithi193 2024-09-16T08:25:58.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:57 smithi193 ceph-mon[25614]: [16/Sep/2024:08:25:56] ENGINE Bus STARTING 2024-09-16T08:25:58.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:57 smithi193 ceph-mon[25614]: Deploying cephadm binary to smithi090 2024-09-16T08:25:58.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:57 smithi193 ceph-mon[25614]: [16/Sep/2024:08:25:56] ENGINE Serving on http://172.21.15.193:8765 2024-09-16T08:25:58.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:57 smithi193 ceph-mon[25614]: [16/Sep/2024:08:25:56] ENGINE Serving on https://172.21.15.193:7150 2024-09-16T08:25:58.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:57 smithi193 ceph-mon[25614]: [16/Sep/2024:08:25:56] ENGINE Bus STARTED 2024-09-16T08:25:58.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:57 smithi193 ceph-mon[25614]: [16/Sep/2024:08:25:56] ENGINE Error in HTTPServer.serve 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: Traceback (most recent call last): 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._run(expiration_interval) 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: s = self.context.wrap_socket( 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:25:58.253 INFO:journalctl@ceph.mon.b.smithi193.stdout: return self.sslsocket_class._create( 2024-09-16T08:25:58.254 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:25:58.254 INFO:journalctl@ceph.mon.b.smithi193.stdout: self.do_handshake() 2024-09-16T08:25:58.254 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:25:58.254 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._sslobj.do_handshake() 2024-09-16T08:25:58.254 INFO:journalctl@ceph.mon.b.smithi193.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:25:58.254 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:57 smithi193 ceph-mon[25614]: mgrmap e25: x(active, since 3s) 2024-09-16T08:25:58.312 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:58.079+0000 7f21b08cc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T08:25:58.947 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph versions' 2024-09-16T08:25:58.994 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:58 smithi193 ceph-mon[25614]: Deploying cephadm binary to smithi193 2024-09-16T08:25:58.995 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:58 smithi193 ceph-mon[25614]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:25:58.995 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:58 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:58.995 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:58 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:58.995 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:58 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:58.995 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:58 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:25:58.995 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:58 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:25:58.995 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:58 smithi193 ceph-mon[25614]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:58.995 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:58 smithi193 ceph-mon[25614]: Standby manager daemon y started 2024-09-16T08:25:59.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[30018]: Deploying cephadm binary to smithi193 2024-09-16T08:25:59.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[30018]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:25:59.181 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:59.182 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:59.182 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:59.182 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:25:59.182 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:25:59.182 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[30018]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:59.182 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[30018]: Standby manager daemon y started 2024-09-16T08:25:59.183 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:58.873+0000 7f21b08cc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T08:25:59.183 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:25:58] ENGINE Bus STARTING 2024-09-16T08:25:59.183 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: CherryPy Checker: 2024-09-16T08:25:59.183 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: The Application mounted at '' has an empty config. 2024-09-16T08:25:59.183 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: 2024-09-16T08:25:59.183 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:25:58] ENGINE Serving on http://:::9283 2024-09-16T08:25:59.183 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: [16/Sep/2024:08:25:58] ENGINE Bus STARTED 2024-09-16T08:25:59.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[23701]: Deploying cephadm binary to smithi193 2024-09-16T08:25:59.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[23701]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:25:59.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:59.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:59.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:25:59.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:25:59.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:25:59.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[23701]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:25:59.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:58 smithi090 ceph-mon[23701]: Standby manager daemon y started 2024-09-16T08:25:59.812 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:25:59] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-16T08:26:00.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T08:26:00.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:26:00.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T08:26:00.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:26:00.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:26:00.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[30018]: mgrmap e26: x(active, since 5s), standbys: y 2024-09-16T08:26:00.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:00.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:00.161 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:00.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T08:26:00.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:26:00.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T08:26:00.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:26:00.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:26:00.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[23701]: mgrmap e26: x(active, since 5s), standbys: y 2024-09-16T08:26:00.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:00.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:00.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:25:59 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:59 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:59 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:59 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:59 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.90:0/2488080994' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:59 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:59 smithi193 ceph-mon[25614]: mgrmap e26: x(active, since 5s), standbys: y 2024-09-16T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:59 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:59 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:00.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:25:59 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:00.554 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:26:01.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:00 smithi090 ceph-mon[30018]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:01.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:00 smithi090 ceph-mon[23701]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:01.166 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:00 smithi193 ceph-mon[25614]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:02.959 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:02 smithi090 ceph-mon[30018]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T08:26:02.960 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:02 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:26:02.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:02 smithi090 ceph-mon[23701]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T08:26:02.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:02 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:26:03.214 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:02 smithi193 ceph-mon[25614]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T08:26:03.214 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:02 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:26:03.441 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:26:03.441 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-09-16T08:26:03.441 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 3 2024-09-16T08:26:03.441 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:26:03.441 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-09-16T08:26:03.441 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 1, 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 8 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw": { 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 16, 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T08:26:03.442 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:26:03.443 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:26:04.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:03 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3026845535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:04.296 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:03 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3026845535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:04.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:03 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3026845535' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:04.319 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-09-16T08:26:04.715 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:26:05.109 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:04 smithi090 ceph-mon[30018]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T08:26:05.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:04 smithi090 ceph-mon[23701]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T08:26:05.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:04 smithi193 ceph-mon[25614]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T08:26:07.167 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:06 smithi090 ceph-mon[30018]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:26:07.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:06 smithi090 ceph-mon[23701]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:26:07.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:06 smithi193 ceph-mon[25614]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:26:07.662 INFO:teuthology.orchestra.run.smithi090.stdout:wait for servicemap items w/ changing names to refresh 2024-09-16T08:26:08.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'sleep 60' 2024-09-16T08:26:08.639 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:26:09.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:08 smithi090 ceph-mon[30018]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:26:09.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:08 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:09.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:08 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:09.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:08 smithi090 ceph-mon[23701]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:26:09.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:08 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:09.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:08 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:09.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:08 smithi193 ceph-mon[25614]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:26:09.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:08 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:09.205 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:08 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:09.990 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:26:09] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-16T08:26:10.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:10 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:10.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:10 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:10.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:10 smithi193 ceph-mon[25614]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:26:10.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:10 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:10.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:10 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:10.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:10.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:10.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[23701]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:26:10.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:10.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[30018]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:26:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:10 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:13.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:12.891Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:26:13.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:12.893Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:26:13.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:12.894Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:26:13.252 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:12.895Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:26:13.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:12.895Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:26:13.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:26:13.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:26:13.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:13.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:13.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.254 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.254 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.254 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.254 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:13.254 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:13.254 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:12 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:13.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:26:13.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:26:13.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:13.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:13.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='client.24634 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:26:13.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:13.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:12 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:14.225 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: Adjusting osd_memory_target on smithi193 to 2736M 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.226 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: Adjusting osd_memory_target on smithi193 to 2736M 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.228 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.228 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.228 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:14.228 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:13 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: Adjusting osd_memory_target on smithi193 to 2736M 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:14.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:14.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:13 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:15.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:15.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:15.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-16T08:26:15.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: Deploying daemon agent.smithi090 on smithi090 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-16T08:26:15.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: Deploying daemon agent.smithi090 on smithi090 2024-09-16T08:26:15.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-16T08:26:15.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:15.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:15.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:15.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:15.064 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:14 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:15.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:15.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:15.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-16T08:26:15.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: Deploying daemon agent.smithi090 on smithi090 2024-09-16T08:26:15.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: Health check failed: 1 Cephadm Agent(s) are not reporting. Hosts may be offline (CEPHADM_AGENT_DOWN) 2024-09-16T08:26:15.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:15.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:15.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:15.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:15.253 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:14 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:16.048 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:15 smithi090 ceph-mon[30018]: Reconfiguring iscsi.foo.smithi090.jxkvvk (dependencies changed)... 2024-09-16T08:26:16.048 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:15 smithi090 ceph-mon[30018]: Reconfiguring daemon iscsi.foo.smithi090.jxkvvk on smithi090 2024-09-16T08:26:16.049 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:15 smithi090 ceph-mon[23701]: Reconfiguring iscsi.foo.smithi090.jxkvvk (dependencies changed)... 2024-09-16T08:26:16.049 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:15 smithi090 ceph-mon[23701]: Reconfiguring daemon iscsi.foo.smithi090.jxkvvk on smithi090 2024-09-16T08:26:16.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:15 smithi193 ceph-mon[25614]: Reconfiguring iscsi.foo.smithi090.jxkvvk (dependencies changed)... 2024-09-16T08:26:16.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:15 smithi193 ceph-mon[25614]: Reconfiguring daemon iscsi.foo.smithi090.jxkvvk on smithi090 2024-09-16T08:26:17.215 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:16 smithi090 ceph-mon[30018]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:26:17.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:16 smithi090 ceph-mon[23701]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:26:17.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:16 smithi193 ceph-mon[25614]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:26:19.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:18 smithi090 ceph-mon[23701]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-16T08:26:19.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:18 smithi090 ceph-mon[30018]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-16T08:26:19.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:18 smithi193 ceph-mon[25614]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-16T08:26:19.939 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: 2024-09-16T08:26:19.631+0000 7f86e6589640 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:19.939 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Creating ceph-iscsi config... 2024-09-16T08:26:19.940 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:19.940 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:19.940 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:19.940 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:19.940 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:19.941 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Traceback (most recent call last): 2024-09-16T08:26:19.941 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:19.941 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return _run_code(code, main_globals, None, 2024-09-16T08:26:19.941 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:19.941 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: exec(code, run_globals) 2024-09-16T08:26:19.941 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:19.942 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:19.942 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:19.942 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:19.942 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:19.942 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:19.942 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:19.943 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:19.943 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:19.944 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Traceback (most recent call last): 2024-09-16T08:26:19.944 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1164, in _check_daemons 2024-09-16T08:26:19.944 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: self.mgr._daemon_action(daemon_spec, action=action) 2024-09-16T08:26:19.944 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/module.py", line 2557, in _daemon_action 2024-09-16T08:26:19.945 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return self.wait_async( 2024-09-16T08:26:19.945 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/module.py", line 878, in wait_async 2024-09-16T08:26:19.945 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return self.event_loop.get_result(coro, timeout) 2024-09-16T08:26:19.945 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-09-16T08:26:19.946 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return future.result(timeout) 2024-09-16T08:26:19.946 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib64/python3.9/concurrent/futures/_base.py", line 446, in result 2024-09-16T08:26:19.946 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return self.__get_result() 2024-09-16T08:26:19.946 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result 2024-09-16T08:26:19.946 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: raise self._exception 2024-09-16T08:26:19.946 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1400, in _create_daemon 2024-09-16T08:26:19.947 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: out, err, code = await self._run_cephadm( 2024-09-16T08:26:19.947 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1743, in _run_cephadm 2024-09-16T08:26:19.947 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: raise OrchestratorError( 2024-09-16T08:26:19.947 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:19.947 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Creating ceph-iscsi config... 2024-09-16T08:26:19.947 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:19.948 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:19.948 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:19.948 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:19.948 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:19.948 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Traceback (most recent call last): 2024-09-16T08:26:19.948 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:19.949 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return _run_code(code, main_globals, None, 2024-09-16T08:26:19.949 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:19.949 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: exec(code, run_globals) 2024-09-16T08:26:19.949 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:19.949 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:19.949 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:19.949 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:19.949 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:19.949 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:19.950 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:19.950 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:19.950 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:20.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:19 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:20.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:19 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:20.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:19 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:20.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:19 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:20.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:19 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[23928]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:26:19] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-16T08:26:20.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:20.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:19 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:21.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:20 smithi193 ceph-mon[25614]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-16T08:26:21.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:20 smithi193 ceph-mon[25614]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:21.002 INFO:journalctl@ceph.mon.b.smithi193.stdout: Creating ceph-iscsi config... 2024-09-16T08:26:21.002 INFO:journalctl@ceph.mon.b.smithi193.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:21.002 INFO:journalctl@ceph.mon.b.smithi193.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:21.002 INFO:journalctl@ceph.mon.b.smithi193.stdout: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:21.002 INFO:journalctl@ceph.mon.b.smithi193.stdout: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: Traceback (most recent call last): 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: return _run_code(code, main_globals, None, 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: exec(code, run_globals) 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:21.003 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: Traceback (most recent call last): 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1164, in _check_daemons 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2557, in _daemon_action 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: return self.wait_async( 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 878, in wait_async 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: return self.event_loop.get_result(coro, timeout) 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: return future.result(timeout) 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/concurrent/futures/_base.py", line 446, in result 2024-09-16T08:26:21.004 INFO:journalctl@ceph.mon.b.smithi193.stdout: return self.__get_result() 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: raise self._exception 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1400, in _create_daemon 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: out, err, code = await self._run_cephadm( 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1743, in _run_cephadm 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: raise OrchestratorError( 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: Creating ceph-iscsi config... 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.005 INFO:journalctl@ceph.mon.b.smithi193.stdout: Traceback (most recent call last): 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: return _run_code(code, main_globals, None, 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: exec(code, run_globals) 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.006 INFO:journalctl@ceph.mon.b.smithi193.stdout: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:20 smithi090 ceph-mon[23701]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-16T08:26:21.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:20 smithi090 ceph-mon[23701]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: Creating ceph-iscsi config... 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: return _run_code(code, main_globals, None, 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: exec(code, run_globals) 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:21.313 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1164, in _check_daemons 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2557, in _daemon_action 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.wait_async( 2024-09-16T08:26:21.314 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 878, in wait_async 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.event_loop.get_result(coro, timeout) 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: return future.result(timeout) 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/concurrent/futures/_base.py", line 446, in result 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.__get_result() 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: raise self._exception 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1400, in _create_daemon 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: out, err, code = await self._run_cephadm( 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1743, in _run_cephadm 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: raise OrchestratorError( 2024-09-16T08:26:21.315 INFO:journalctl@ceph.mon.a.smithi090.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: Creating ceph-iscsi config... 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: return _run_code(code, main_globals, None, 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: exec(code, run_globals) 2024-09-16T08:26:21.316 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.a.smithi090.stdout: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.a.smithi090.stdout: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:20 smithi090 ceph-mon[30018]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 0 op/s 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:20 smithi090 ceph-mon[30018]: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:21.317 INFO:journalctl@ceph.mon.c.smithi090.stdout: Creating ceph-iscsi config... 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: return _run_code(code, main_globals, None, 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: exec(code, run_globals) 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:21.318 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1164, in _check_daemons 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: self.mgr._daemon_action(daemon_spec, action=action) 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 2557, in _daemon_action 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: return self.wait_async( 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/module.py", line 878, in wait_async 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: return self.event_loop.get_result(coro, timeout) 2024-09-16T08:26:21.319 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: return future.result(timeout) 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/concurrent/futures/_base.py", line 446, in result 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: return self.__get_result() 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: raise self._exception 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1400, in _create_daemon 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: out, err, code = await self._run_cephadm( 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1743, in _run_cephadm 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: raise OrchestratorError( 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: Creating ceph-iscsi config... 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:21.320 INFO:journalctl@ceph.mon.c.smithi090.stdout: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: return _run_code(code, main_globals, None, 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: exec(code, run_globals) 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:21.321 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:21.322 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:21.322 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:21.322 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:21.322 INFO:journalctl@ceph.mon.c.smithi090.stdout: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:21.322 INFO:journalctl@ceph.mon.c.smithi090.stdout: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:22.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:21 smithi090 ceph-mon[30018]: Deploying daemon agent.smithi193 on smithi193 2024-09-16T08:26:22.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:21 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:22.062 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:21 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:22.063 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:21 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:21 smithi090 ceph-mon[23701]: Deploying daemon agent.smithi193 on smithi193 2024-09-16T08:26:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:21 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:21 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:22.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:21 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:22.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:21 smithi193 ceph-mon[25614]: Deploying daemon agent.smithi193 on smithi193 2024-09-16T08:26:22.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:21 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:22.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:21 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:22.203 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:21 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:23.228 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:22 smithi193 ceph-mon[25614]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:26:23.228 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:22 smithi193 ceph-mon[25614]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:26:23.228 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:22 smithi193 ceph-mon[25614]: Reconfiguring daemon prometheus.a on smithi193 2024-09-16T08:26:23.228 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:22 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:23.228 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:22 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:23.229 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:22 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[30018]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:26:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[30018]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:26:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[30018]: Reconfiguring daemon prometheus.a on smithi193 2024-09-16T08:26:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:23.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:23.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[23701]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:26:23.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[23701]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:26:23.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[23701]: Reconfiguring daemon prometheus.a on smithi193 2024-09-16T08:26:23.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:23.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:23.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:22 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:24.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:23 smithi193 ceph-mon[25614]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-09-16T08:26:24.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:23 smithi193 ceph-mon[25614]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:24.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:23 smithi090 ceph-mon[30018]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-09-16T08:26:24.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:23 smithi090 ceph-mon[30018]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:24.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:23 smithi090 ceph-mon[23701]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-09-16T08:26:24.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:23 smithi090 ceph-mon[23701]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:26.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:25 smithi193 ceph-mon[25614]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 806 B/s rd, 0 op/s 2024-09-16T08:26:26.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:25 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:26.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:25 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:26.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:25 smithi090 ceph-mon[30018]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 806 B/s rd, 0 op/s 2024-09-16T08:26:26.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:25 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:26.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:25 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:26.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:25 smithi090 ceph-mon[23701]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 806 B/s rd, 0 op/s 2024-09-16T08:26:26.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:25 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:26.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:25 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:27.502 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 systemd[1]: Stopping Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:26:27.790 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T08:26:27.790 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-16T08:26:27.790 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-16T08:26:27.790 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T08:26:27.790 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-16T08:26:27.790 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-16T08:26:27.790 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T08:26:27.790 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-16T08:26:27.791 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-16T08:26:27.791 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T08:26:27.791 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-16T08:26:27.791 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[52692]: ts=2024-09-16T08:26:27.708Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-16T08:26:28.163 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:27 smithi193 ceph-mon[25614]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 358 B/s rd, 0 op/s 2024-09-16T08:26:28.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:27 smithi090 ceph-mon[30018]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 358 B/s rd, 0 op/s 2024-09-16T08:26:28.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:27 smithi090 ceph-mon[23701]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 358 B/s rd, 0 op/s 2024-09-16T08:26:28.502 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:28 smithi193 podman[78086]: 2024-09-16 08:26:28.301055677 +0000 UTC m=+0.957235270 container cleanup 0416160d55ca4337bb9e5160aae3d3731563c9caf5deed41b77dd35208f01173 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:28.502 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:28 smithi193 bash[78086]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T08:26:28.950 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:28 smithi193 podman[78137]: 2024-09-16 08:26:28.560684668 +0000 UTC m=+0.844462827 container remove 0416160d55ca4337bb9e5160aae3d3731563c9caf5deed41b77dd35208f01173 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:30.002 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:29 smithi193 ceph-mon[25614]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 358 B/s rd, 0 op/s 2024-09-16T08:26:30.002 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:29 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-16T08:26:30.002 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:29 smithi193 systemd[1]: Stopped Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:26:30.002 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:29 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 2.361s CPU time. 2024-09-16T08:26:30.002 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:29 smithi193 systemd[1]: Starting Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:26:30.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:29 smithi090 ceph-mon[23701]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 358 B/s rd, 0 op/s 2024-09-16T08:26:30.158 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:29 smithi090 ceph-mon[30018]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 358 B/s rd, 0 op/s 2024-09-16T08:26:30.419 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 podman[78429]: 2024-09-16 08:26:30.109041435 +0000 UTC m=+0.017413135 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-16T08:26:30.419 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 podman[78429]: 2024-09-16 08:26:30.233779112 +0000 UTC m=+0.142150842 container create 54a1e828f158b83da723f72dbdda465e8b4a19ad74edc319a65fd5b64ab15024 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:30.744 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 podman[78429]: 2024-09-16 08:26:30.419184432 +0000 UTC m=+0.327556120 container init 54a1e828f158b83da723f72dbdda465e8b4a19ad74edc319a65fd5b64ab15024 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:30.744 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 podman[78429]: 2024-09-16 08:26:30.42293552 +0000 UTC m=+0.331307211 container start 54a1e828f158b83da723f72dbdda465e8b4a19ad74edc319a65fd5b64ab15024 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:30.744 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.447Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-16T08:26:30.744 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.447Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-16T08:26:30.744 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.447Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi193 (none))" 2024-09-16T08:26:30.744 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.447Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T08:26:30.744 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.447Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T08:26:30.744 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.449Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.449Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.450Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.450Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.453Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.453Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.851µs 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.453Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.454Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=2 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.464Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=2 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.464Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=2 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.464Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=26.557µs wal_replay_duration=11.661512ms wbl_replay_duration=110ns total_replay_duration=11.707497ms 2024-09-16T08:26:30.745 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.466Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T08:26:30.746 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.466Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-16T08:26:30.746 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.466Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T08:26:30.746 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.487Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=21.278197ms db_storage=760ns remote_storage=999ns web_handler=371ns query_engine=480ns scrape=387.319µs scrape_sd=114.43µs notify=31.467µs notify_sd=19.49µs rules=20.363002ms tracing=5.857µs 2024-09-16T08:26:30.746 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.487Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-16T08:26:30.746 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:30.487Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T08:26:30.746 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 bash[78429]: 54a1e828f158b83da723f72dbdda465e8b4a19ad74edc319a65fd5b64ab15024 2024-09-16T08:26:30.746 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:30 smithi193 systemd[1]: Started Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:26:31.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 372 B/s rd, 0 op/s 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: Upgrade: Need to upgrade myself (mgr.x) 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[30018]: Upgrade: Need to upgrade myself (mgr.x) 2024-09-16T08:26:31.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 372 B/s rd, 0 op/s 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: Upgrade: Need to upgrade myself (mgr.x) 2024-09-16T08:26:31.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:31 smithi090 ceph-mon[23701]: Upgrade: Need to upgrade myself (mgr.x) 2024-09-16T08:26:31.417 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.417 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.417 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 372 B/s rd, 0 op/s 2024-09-16T08:26:31.418 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:31.418 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.418 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.418 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:31.418 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:31.418 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:31.418 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:31.418 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: Upgrade: Need to upgrade myself (mgr.x) 2024-09-16T08:26:31.418 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:31 smithi193 ceph-mon[25614]: Upgrade: Need to upgrade myself (mgr.x) 2024-09-16T08:26:32.169 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:32 smithi193 ceph-mon[25614]: Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:32.169 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:32 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/104607113' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T08:26:32.169 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:32 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2163067609' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2381174981"}]: dispatch 2024-09-16T08:26:32.169 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:32 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2381174981"}]: dispatch 2024-09-16T08:26:32.537 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:32 smithi090 ceph-mon[30018]: Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:32.537 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:32 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/104607113' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T08:26:32.537 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:32 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2163067609' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2381174981"}]: dispatch 2024-09-16T08:26:32.537 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:32 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2381174981"}]: dispatch 2024-09-16T08:26:32.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:32 smithi090 ceph-mon[23701]: Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:32.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:32 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/104607113' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T08:26:32.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:32 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2163067609' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2381174981"}]: dispatch 2024-09-16T08:26:32.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:32 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2381174981"}]: dispatch 2024-09-16T08:26:33.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:33.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2381174981"}]': finished 2024-09-16T08:26:33.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: osdmap e97: 8 total, 8 up, 8 in 2024-09-16T08:26:33.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: Upgrade: Updating mgr.y 2024-09-16T08:26:33.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.487 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: Deploying daemon mgr.y on smithi090 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi193 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2449392369' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/960941707"}]: dispatch 2024-09-16T08:26:33.488 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/960941707"}]: dispatch 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2381174981"}]': finished 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: osdmap e97: 8 total, 8 up, 8 in 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: Upgrade: Updating mgr.y 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: Deploying daemon mgr.y on smithi090 2024-09-16T08:26:33.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:33.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:33.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi193 2024-09-16T08:26:33.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2449392369' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/960941707"}]: dispatch 2024-09-16T08:26:33.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:33 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/960941707"}]: dispatch 2024-09-16T08:26:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2381174981"}]': finished 2024-09-16T08:26:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: osdmap e97: 8 total, 8 up, 8 in 2024-09-16T08:26:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: Upgrade: Updating mgr.y 2024-09-16T08:26:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:26:33.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:33.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: Deploying daemon mgr.y on smithi090 2024-09-16T08:26:33.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:33.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:33.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi193 2024-09-16T08:26:33.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:33.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2449392369' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/960941707"}]: dispatch 2024-09-16T08:26:33.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:33 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/960941707"}]: dispatch 2024-09-16T08:26:34.039 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:33 smithi090 systemd[1]: Stopping Ceph mgr.y for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:26:34.305 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:34 smithi090 podman[72074]: 2024-09-16 08:26:34.038712854 +0000 UTC m=+0.110523257 container died 92b236ae51c08f45e454c3c46957c820b9993f2ff2d91b197b032c09d0795668 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T08:26:34.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:34 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/960941707"}]': finished 2024-09-16T08:26:34.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:34 smithi090 ceph-mon[30018]: osdmap e98: 8 total, 8 up, 8 in 2024-09-16T08:26:34.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:34 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3710364611' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/3507480917"}]: dispatch 2024-09-16T08:26:34.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:34 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/960941707"}]': finished 2024-09-16T08:26:34.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:34 smithi090 ceph-mon[23701]: osdmap e98: 8 total, 8 up, 8 in 2024-09-16T08:26:34.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:34 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3710364611' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/3507480917"}]: dispatch 2024-09-16T08:26:34.563 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:34 smithi090 podman[72074]: 2024-09-16 08:26:34.383262579 +0000 UTC m=+0.455072971 container cleanup 92b236ae51c08f45e454c3c46957c820b9993f2ff2d91b197b032c09d0795668 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373) 2024-09-16T08:26:34.563 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:34 smithi090 bash[72074]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y 2024-09-16T08:26:34.563 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:34 smithi090 podman[72087]: 2024-09-16 08:26:34.543300934 +0000 UTC m=+0.499123975 container remove 92b236ae51c08f45e454c3c46957c820b9993f2ff2d91b197b032c09d0795668 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:26:34.563 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:34 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-16T08:26:34.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:34 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/960941707"}]': finished 2024-09-16T08:26:34.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:34 smithi193 ceph-mon[25614]: osdmap e98: 8 total, 8 up, 8 in 2024-09-16T08:26:34.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:34 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3710364611' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/3507480917"}]: dispatch 2024-09-16T08:26:35.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:34 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-16T08:26:35.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:34 smithi090 systemd[1]: Stopped Ceph mgr.y for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:26:35.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:34 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y.service: Consumed 57.268s CPU time. 2024-09-16T08:26:35.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:34 smithi090 systemd[1]: Starting Ceph mgr.y for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:26:35.442 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:35 smithi090 ceph-mon[23701]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:35.442 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:35 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3710364611' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/3507480917"}]': finished 2024-09-16T08:26:35.442 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:35 smithi090 ceph-mon[23701]: osdmap e99: 8 total, 8 up, 8 in 2024-09-16T08:26:35.442 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:35 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3586248686' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/3507480917"}]: dispatch 2024-09-16T08:26:35.442 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:35 smithi090 podman[72238]: 2024-09-16 08:26:35.072450369 +0000 UTC m=+0.015930899 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T08:26:35.442 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:35 smithi090 podman[72238]: 2024-09-16 08:26:35.196534898 +0000 UTC m=+0.140015418 container create fa011575ebeebc942d838bc1db10a329124aeb5da069379bd3a24250804a356a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T08:26:35.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:35 smithi090 ceph-mon[30018]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:35.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:35 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3710364611' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/3507480917"}]': finished 2024-09-16T08:26:35.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:35 smithi090 ceph-mon[30018]: osdmap e99: 8 total, 8 up, 8 in 2024-09-16T08:26:35.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:35 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3586248686' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/3507480917"}]: dispatch 2024-09-16T08:26:35.707 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:35 smithi090 podman[72238]: 2024-09-16 08:26:35.442124709 +0000 UTC m=+0.385605234 container init fa011575ebeebc942d838bc1db10a329124aeb5da069379bd3a24250804a356a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T08:26:35.708 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:35 smithi090 podman[72238]: 2024-09-16 08:26:35.446522341 +0000 UTC m=+0.390002867 container start fa011575ebeebc942d838bc1db10a329124aeb5da069379bd3a24250804a356a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:26:35.708 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:35 smithi090 ceph-mgr[72278]: -- 172.21.15.90:0/367895152 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae053950e0 con 0x55ae05373000 2024-09-16T08:26:35.708 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:35 smithi090 bash[72238]: fa011575ebeebc942d838bc1db10a329124aeb5da069379bd3a24250804a356a 2024-09-16T08:26:35.708 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:35 smithi090 systemd[1]: Started Ceph mgr.y for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:26:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:35 smithi193 ceph-mon[25614]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:35 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3710364611' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6801/3507480917"}]': finished 2024-09-16T08:26:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:35 smithi193 ceph-mon[25614]: osdmap e99: 8 total, 8 up, 8 in 2024-09-16T08:26:35.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:35 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3586248686' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/3507480917"}]: dispatch 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3586248686' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/3507480917"}]': finished 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: osdmap e100: 8 total, 8 up, 8 in 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1508312095' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/4275095857"}]: dispatch 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:26:36.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:26:36.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-16T08:26:36.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:36.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:36.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: Reconfiguring iscsi.foo.smithi090.jxkvvk (dependencies changed)... 2024-09-16T08:26:36.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:36.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:36.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:36.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[23701]: Reconfiguring daemon iscsi.foo.smithi090.jxkvvk on smithi090 2024-09-16T08:26:36.639 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3586248686' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/3507480917"}]': finished 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: osdmap e100: 8 total, 8 up, 8 in 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1508312095' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/4275095857"}]: dispatch 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:36.640 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:36.641 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: Reconfiguring iscsi.foo.smithi090.jxkvvk (dependencies changed)... 2024-09-16T08:26:36.641 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:36.641 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:36.641 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:36.641 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:36 smithi090 ceph-mon[30018]: Reconfiguring daemon iscsi.foo.smithi090.jxkvvk on smithi090 2024-09-16T08:26:36.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3586248686' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:6800/3507480917"}]': finished 2024-09-16T08:26:36.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: osdmap e100: 8 total, 8 up, 8 in 2024-09-16T08:26:36.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1508312095' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/4275095857"}]: dispatch 2024-09-16T08:26:36.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:36.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:36.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2 op/s 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: Reconfiguring iscsi.foo.smithi090.jxkvvk (dependencies changed)... 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi090.jxkvvk", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:36.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:36 smithi193 ceph-mon[25614]: Reconfiguring daemon iscsi.foo.smithi090.jxkvvk on smithi090 2024-09-16T08:26:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1508312095' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/4275095857"}]': finished 2024-09-16T08:26:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[23701]: osdmap e101: 8 total, 8 up, 8 in 2024-09-16T08:26:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3222618467' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2913896472"}]: dispatch 2024-09-16T08:26:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2913896472"}]: dispatch 2024-09-16T08:26:37.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[23701]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-16T08:26:37.712 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1508312095' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/4275095857"}]': finished 2024-09-16T08:26:37.712 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[30018]: osdmap e101: 8 total, 8 up, 8 in 2024-09-16T08:26:37.712 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3222618467' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2913896472"}]: dispatch 2024-09-16T08:26:37.712 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2913896472"}]: dispatch 2024-09-16T08:26:37.712 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:37 smithi090 ceph-mon[30018]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-16T08:26:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:37 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1508312095' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/4275095857"}]': finished 2024-09-16T08:26:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:37 smithi193 ceph-mon[25614]: osdmap e101: 8 total, 8 up, 8 in 2024-09-16T08:26:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:37 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3222618467' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2913896472"}]: dispatch 2024-09-16T08:26:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:37 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2913896472"}]: dispatch 2024-09-16T08:26:37.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:37 smithi193 ceph-mon[25614]: Health check cleared: CEPHADM_AGENT_DOWN (was: 1 Cephadm Agent(s) are not reporting. Hosts may be offline) 2024-09-16T08:26:38.603 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2913896472"}]': finished 2024-09-16T08:26:38.603 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[30018]: osdmap e102: 8 total, 8 up, 8 in 2024-09-16T08:26:38.603 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/17123742' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2594198040"}]: dispatch 2024-09-16T08:26:38.603 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2594198040"}]: dispatch 2024-09-16T08:26:38.604 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[30018]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:38.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2913896472"}]': finished 2024-09-16T08:26:38.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[23701]: osdmap e102: 8 total, 8 up, 8 in 2024-09-16T08:26:38.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/17123742' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2594198040"}]: dispatch 2024-09-16T08:26:38.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2594198040"}]: dispatch 2024-09-16T08:26:38.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:38 smithi090 ceph-mon[23701]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:38 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2913896472"}]': finished 2024-09-16T08:26:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:38 smithi193 ceph-mon[25614]: osdmap e102: 8 total, 8 up, 8 in 2024-09-16T08:26:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:38 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/17123742' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2594198040"}]: dispatch 2024-09-16T08:26:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:38 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2594198040"}]: dispatch 2024-09-16T08:26:38.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:38 smithi193 ceph-mon[25614]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:39.703 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:39 smithi090 ceph-mon[30018]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2594198040"}]': finished 2024-09-16T08:26:39.703 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:39 smithi090 ceph-mon[30018]: osdmap e103: 8 total, 8 up, 8 in 2024-09-16T08:26:39.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:39 smithi090 ceph-mon[23701]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2594198040"}]': finished 2024-09-16T08:26:39.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:39 smithi090 ceph-mon[23701]: osdmap e103: 8 total, 8 up, 8 in 2024-09-16T08:26:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:39 smithi193 ceph-mon[25614]: from='client.? ' entity='client.iscsi.foo.smithi090.jxkvvk' cmd='[{"prefix": "osd blocklist", "blocklistop": "rm", "addr": "172.21.15.90:0/2594198040"}]': finished 2024-09-16T08:26:39.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:39 smithi193 ceph-mon[25614]: osdmap e103: 8 total, 8 up, 8 in 2024-09-16T08:26:40.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-mon[30018]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:40.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:40.474 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:40.475 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-mon[23701]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:40.475 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:40.475 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:40.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:40 smithi193 ceph-mon[25614]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:40.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:40 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' 2024-09-16T08:26:40.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:40 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:41.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:26:40] ENGINE Bus STARTING 2024-09-16T08:26:41.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: CherryPy Checker: 2024-09-16T08:26:41.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: The Application mounted at '' has an empty config. 2024-09-16T08:26:41.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: 2024-09-16T08:26:41.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:26:40] ENGINE Serving on http://:::9283 2024-09-16T08:26:41.063 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:40 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:26:40] ENGINE Bus STARTED 2024-09-16T08:26:41.348 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:26:41] ENGINE Bus STOPPING 2024-09-16T08:26:41.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: Standby manager daemon y restarted 2024-09-16T08:26:41.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: Standby manager daemon y started 2024-09-16T08:26:41.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T08:26:41.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:26:41.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T08:26:41.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:26:41.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:41.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-16T08:26:41.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-16T08:26:41.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: Activating manager daemon y 2024-09-16T08:26:41.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: osdmap e104: 8 total, 8 up, 8 in 2024-09-16T08:26:41.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.24859 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-16T08:26:41.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: mgrmap e27: y(active, starting, since 0.00785482s) 2024-09-16T08:26:41.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:26:41.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:26:41.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:26:41.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:26:41.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:26:41.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:26:41.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:26:41.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:26:41.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:26:41.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:26:41.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:26:41.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:26:41.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:26:41.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:26:41.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:26:41.508 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: 2024-09-16T08:26:41.142+0000 7f86e6589640 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:41.508 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Creating ceph-iscsi config... 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Traceback (most recent call last): 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return _run_code(code, main_globals, None, 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: exec(code, run_globals) 2024-09-16T08:26:41.509 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Traceback (most recent call last): 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1164, in _check_daemons 2024-09-16T08:26:41.510 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: self.mgr._daemon_action(daemon_spec, action=action) 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/module.py", line 2557, in _daemon_action 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return self.wait_async( 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/module.py", line 878, in wait_async 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return self.event_loop.get_result(coro, timeout) 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/ssh.py", line 136, in get_result 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return future.result(timeout) 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib64/python3.9/concurrent/futures/_base.py", line 446, in result 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return self.__get_result() 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: raise self._exception 2024-09-16T08:26:41.511 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1400, in _create_daemon 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: out, err, code = await self._run_cephadm( 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1743, in _run_cephadm 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: raise OrchestratorError( 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr: Reconfig daemon iscsi.foo.smithi090.jxkvvk ... 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Creating ceph-iscsi config... 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/iscsi-gateway.cfg 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Write file: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/iscsi.foo.smithi090.jxkvvk/tcmu-runner-entrypoint.sh 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Non-zero exit code 1 from systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: systemctl: stderr Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:41.512 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: systemctl: stderr See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: Traceback (most recent call last): 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: return _run_code(code, main_globals, None, 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: exec(code, run_globals) 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5628, in 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 5616, in main 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3090, in command_deploy_from 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3125, in _common_deploy 2024-09-16T08:26:41.513 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 3145, in _deploy_daemon_container 2024-09-16T08:26:41.514 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/__main__.py", line 1147, in deploy_daemon 2024-09-16T08:26:41.514 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: File "/tmp/tmpwdcnramk.cephadm.build/app/cephadmlib/call_wrappers.py", line 307, in call_throws 2024-09-16T08:26:41.514 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: RuntimeError: Failed command: systemctl restart ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk: Job for ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service failed because the control process exited with error code. 2024-09-16T08:26:41.514 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: See "systemctl status ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" and "journalctl -xeu ceph-8b378aa4-7403-11ef-bceb-c7b262605968@iscsi.foo.smithi090.jxkvvk.service" for details. 2024-09-16T08:26:41.514 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: 2024-09-16T08:26:41.192+0000 7f87522bf640 -1 mgr handle_mgr_map I was active but no longer am 2024-09-16T08:26:41.514 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: ignoring --setuser ceph since I am not root 2024-09-16T08:26:41.514 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: ignoring --setgroup ceph since I am not root 2024-09-16T08:26:41.514 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:41 smithi193 ceph-mgr[68211]: -- 172.21.15.193:0/4107351664 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5dff2f0e0 con 0x55a5dff0c800 2024-09-16T08:26:41.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: Standby manager daemon y restarted 2024-09-16T08:26:41.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: Standby manager daemon y started 2024-09-16T08:26:41.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T08:26:41.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:26:41.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T08:26:41.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:26:41.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:41.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-16T08:26:41.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-16T08:26:41.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: Activating manager daemon y 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: osdmap e104: 8 total, 8 up, 8 in 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.24859 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: mgrmap e27: y(active, starting, since 0.00785482s) 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:26:41.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:26:41.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:26:41.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:26:41.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:26:41.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:26:41.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:26:41.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:26:41.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:26:41.815 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:26:41] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:26:41.815 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:26:41] ENGINE Bus STOPPED 2024-09-16T08:26:41.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: Standby manager daemon y restarted 2024-09-16T08:26:41.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: Standby manager daemon y started 2024-09-16T08:26:41.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T08:26:41.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:26:41.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T08:26:41.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:26:41.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:41.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.24859 172.21.15.193:0/719145861' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd=[{"prefix": "mgr fail", "who": "x"}]: dispatch 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: Activating manager daemon y 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: osdmap e104: 8 total, 8 up, 8 in 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.24859 ' entity='mgr.x' cmd='[{"prefix": "mgr fail", "who": "x"}]': finished 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: mgrmap e27: y(active, starting, since 0.00785482s) 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:26:41.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:26:41.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:26:41.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:26:41.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T08:26:41.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T08:26:41.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:26:41.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:26:41.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:41 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:26:42.381 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: Manager daemon y is now available 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:26:42.382 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: Manager daemon y is now available 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:26:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:42.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:26:42.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:26:42.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:26:42.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:26:42.656 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: Manager daemon y is now available 2024-09-16T08:26:42.656 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:42.656 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:42.657 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:42.657 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi193", "caps": []}]: dispatch 2024-09-16T08:26:42.657 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:42.657 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi090", "caps": []}]: dispatch 2024-09-16T08:26:42.657 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:26:42.658 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:42.658 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:26:42.658 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:26:42.658 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:26:42.658 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:42 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-16T08:26:42.711 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:26:42] ENGINE Bus STARTING 2024-09-16T08:26:42.712 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:26:42] ENGINE Serving on http://:::9283 2024-09-16T08:26:42.712 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:42 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:26:42] ENGINE Bus STARTED 2024-09-16T08:26:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: mgrmap e28: y(active, since 1.17692s) 2024-09-16T08:26:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:26:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]': finished 2024-09-16T08:26:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:43.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: mgrmap e28: y(active, since 1.17692s) 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]': finished 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:43.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:43.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:43.611 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: mgrmap e28: y(active, since 1.17692s) 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: pgmap v3: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]': finished 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:43.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:43 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:43.967 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: 2024-09-16T08:26:43.771+0000 7f41d8bd9640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:08:26:43] ENGINE Error in HTTPServer.serve 2024-09-16T08:26:43.967 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: Traceback (most recent call last): 2024-09-16T08:26:43.967 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:26:43.967 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: self._connections.run(self.expiration_interval) 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: self._run(expiration_interval) 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: s = self.context.wrap_socket( 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: return self.sslsocket_class._create( 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:26:43.968 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: self.do_handshake() 2024-09-16T08:26:43.969 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:26:43.969 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: self._sslobj.do_handshake() 2024-09-16T08:26:43.969 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:26:43.969 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: 2024-09-16T08:26:44.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:26:44.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:26:44.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:44.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 2 failed cephadm daemon(s)) 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: mgrmap e29: y(active, since 2s) 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 2 failed cephadm daemon(s)) 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: mgrmap e29: y(active, since 2s) 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.564 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:44 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-16T08:26:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: Updating smithi193:/etc/ceph/ceph.conf 2024-09-16T08:26:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: pgmap v4: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 2 failed cephadm daemon(s)) 2024-09-16T08:26:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:26:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: mgrmap e29: y(active, since 2s) 2024-09-16T08:26:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:44.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:44 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: [16/Sep/2024:08:26:43] ENGINE Bus STARTING 2024-09-16T08:26:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:45.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: [16/Sep/2024:08:26:43] ENGINE Serving on http://172.21.15.90:8765 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: [16/Sep/2024:08:26:43] ENGINE Serving on https://172.21.15.90:7150 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: [16/Sep/2024:08:26:43] ENGINE Bus STARTED 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: [16/Sep/2024:08:26:43] ENGINE Error in HTTPServer.serve 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout: Traceback (most recent call last): 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._run(expiration_interval) 2024-09-16T08:26:45.503 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: s = self.context.wrap_socket( 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: return self.sslsocket_class._create( 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: self.do_handshake() 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:26:45.504 INFO:journalctl@ceph.mon.b.smithi193.stdout: self._sslobj.do_handshake() 2024-09-16T08:26:45.505 INFO:journalctl@ceph.mon.b.smithi193.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:26:45.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:45.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:45.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:45.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:45.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:26:45.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Reconfiguring daemon prometheus.a on smithi193 2024-09-16T08:26:45.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:45 smithi193 ceph-mon[25614]: Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:45.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: [16/Sep/2024:08:26:43] ENGINE Bus STARTING 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: [16/Sep/2024:08:26:43] ENGINE Serving on http://172.21.15.90:8765 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: [16/Sep/2024:08:26:43] ENGINE Serving on https://172.21.15.90:7150 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: [16/Sep/2024:08:26:43] ENGINE Bus STARTED 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: [16/Sep/2024:08:26:43] ENGINE Error in HTTPServer.serve 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:26:45.671 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._run(expiration_interval) 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: s = self.context.wrap_socket( 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:26:45.672 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.sslsocket_class._create( 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout: self.do_handshake() 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._sslobj.do_handshake() 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Reconfiguring daemon prometheus.a on smithi193 2024-09-16T08:26:45.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[23701]: Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:45.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: [16/Sep/2024:08:26:43] ENGINE Bus STARTING 2024-09-16T08:26:45.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:45.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:26:45.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: [16/Sep/2024:08:26:43] ENGINE Serving on http://172.21.15.90:8765 2024-09-16T08:26:45.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:45.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:26:45.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: [16/Sep/2024:08:26:43] ENGINE Serving on https://172.21.15.90:7150 2024-09-16T08:26:45.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: [16/Sep/2024:08:26:43] ENGINE Bus STARTED 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: [16/Sep/2024:08:26:43] ENGINE Error in HTTPServer.serve 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: Traceback (most recent call last): 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: self._run(expiration_interval) 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:26:45.675 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout: s = self.context.wrap_socket( 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout: return self.sslsocket_class._create( 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout: self.do_handshake() 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout: self._sslobj.do_handshake() 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Updating smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Updating smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: pgmap v5: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:45.676 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T08:26:45.677 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Reconfiguring daemon prometheus.a on smithi193 2024-09-16T08:26:45.677 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:45 smithi090 ceph-mon[30018]: Health check failed: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:46.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[30018]: mgrmap e30: y(active, since 4s) 2024-09-16T08:26:46.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.563 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[23701]: mgrmap e30: y(active, since 4s) 2024-09-16T08:26:46.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:46 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.690 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-mon[25614]: mgrmap e30: y(active, since 4s) 2024-09-16T08:26:46.690 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:26:46] ENGINE Bus STARTING 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: CherryPy Checker: 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: The Application mounted at '' has an empty config. 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: 2024-09-16T08:26:46.691 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:26:46] ENGINE Serving on http://:::9283 2024-09-16T08:26:46.692 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:26:46 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[68186]: [16/Sep/2024:08:26:46] ENGINE Bus STARTED 2024-09-16T08:26:47.663 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: Standby manager daemon x started 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:47.664 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:47 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:47.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: Standby manager daemon x started 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:47.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: pgmap v6: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: pgmap v7: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: pgmap v8: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: pgmap v9: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: Standby manager daemon x started 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/238254788' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:47.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:47.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:47 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:48.670 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:48 smithi193 ceph-mon[25614]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:48.670 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:48 smithi193 ceph-mon[25614]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:48.670 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:48 smithi193 ceph-mon[25614]: mgrmap e31: y(active, since 6s), standbys: x 2024-09-16T08:26:48.670 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:48 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:26:48.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:48 smithi090 ceph-mon[30018]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:48.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:48 smithi090 ceph-mon[30018]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:48.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:48 smithi090 ceph-mon[30018]: mgrmap e31: y(active, since 6s), standbys: x 2024-09-16T08:26:48.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:48 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:26:48.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:48 smithi090 ceph-mon[23701]: pgmap v10: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:48.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:48 smithi090 ceph-mon[23701]: pgmap v11: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:48.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:48 smithi090 ceph-mon[23701]: mgrmap e31: y(active, since 6s), standbys: x 2024-09-16T08:26:48.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:48 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T08:26:50.289 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:50 smithi193 systemd[1]: Stopping Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:26:50.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[30018]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-16T08:26:50.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:50.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:50.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[30018]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 21 op/s 2024-09-16T08:26:50.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[30018]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 0 B/s wr, 23 op/s 2024-09-16T08:26:50.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:50.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[23701]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-16T08:26:50.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:50.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:50.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[23701]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 21 op/s 2024-09-16T08:26:50.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[23701]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 0 B/s wr, 23 op/s 2024-09-16T08:26:50.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:50 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:50.622 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:50 smithi193 ceph-mon[25614]: pgmap v12: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 0 B/s wr, 26 op/s 2024-09-16T08:26:50.622 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:50 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:50.622 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:50 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:50.622 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:50 smithi193 ceph-mon[25614]: pgmap v13: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 21 op/s 2024-09-16T08:26:50.622 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:50 smithi193 ceph-mon[25614]: pgmap v14: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 0 B/s wr, 23 op/s 2024-09-16T08:26:50.622 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:50 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:51.410 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T08:26:51.410 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-16T08:26:51.410 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[78456]: ts=2024-09-16T08:26:51.035Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-16T08:26:51.411 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:51 smithi193 podman[79895]: 2024-09-16 08:26:51.045411785 +0000 UTC m=+0.093494643 container died 54a1e828f158b83da723f72dbdda465e8b4a19ad74edc319a65fd5b64ab15024 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:52.415 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:52 smithi090 ceph-mon[30018]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T08:26:52.416 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:52 smithi090 ceph-mon[23701]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T08:26:52.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:52 smithi193 ceph-mon[25614]: pgmap v15: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 35 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-16T08:26:52.502 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:52 smithi193 podman[79895]: 2024-09-16 08:26:52.121599799 +0000 UTC m=+1.169682651 container cleanup 54a1e828f158b83da723f72dbdda465e8b4a19ad74edc319a65fd5b64ab15024 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:52.502 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:52 smithi193 bash[79895]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T08:26:52.503 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:52 smithi193 podman[79919]: 2024-09-16 08:26:52.348200205 +0000 UTC m=+1.304838375 container remove 54a1e828f158b83da723f72dbdda465e8b4a19ad74edc319a65fd5b64ab15024 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:53.585 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1888792622' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T08:26:53.585 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-16T08:26:53.585 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 systemd[1]: Stopped Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:26:53.585 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.027s CPU time. 2024-09-16T08:26:53.585 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 systemd[1]: Starting Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:26:53.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:53 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1888792622' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T08:26:53.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:53 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1888792622' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T08:26:53.871 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 podman[80225]: 2024-09-16 08:26:53.488811779 +0000 UTC m=+0.020115113 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-16T08:26:53.871 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 podman[80225]: 2024-09-16 08:26:53.644531635 +0000 UTC m=+0.175835000 container create c0c16dcd857e5a0bdc076fc7371a8e45466048d11d0760e559706de1c771f75b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:53.871 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 podman[80225]: 2024-09-16 08:26:53.871636809 +0000 UTC m=+0.402940137 container init c0c16dcd857e5a0bdc076fc7371a8e45466048d11d0760e559706de1c771f75b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:54.158 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 podman[80225]: 2024-09-16 08:26:53.875227414 +0000 UTC m=+0.406530738 container start c0c16dcd857e5a0bdc076fc7371a8e45466048d11d0760e559706de1c771f75b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:26:54.158 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.898Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-16T08:26:54.158 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.898Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.898Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi193 (none))" 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.898Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.898Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.900Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.901Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.901Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.901Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.903Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.903Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.27µs 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.903Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.904Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=3 2024-09-16T08:26:54.159 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.913Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=3 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.914Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=3 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.914Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=3 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.914Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.547µs wal_replay_duration=11.577173ms wbl_replay_duration=252ns total_replay_duration=11.612152ms 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.917Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.917Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.917Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.935Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=18.438108ms db_storage=1.25µs remote_storage=977ns web_handler=425ns query_engine=672ns scrape=399.082µs scrape_sd=125.934µs notify=17.153µs notify_sd=18.843µs rules=17.526845ms tracing=5.111µs 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.935Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:26:53.935Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 bash[80225]: c0c16dcd857e5a0bdc076fc7371a8e45466048d11d0760e559706de1c771f75b 2024-09-16T08:26:54.160 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:26:53 smithi193 systemd[1]: Started Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:26:54.442 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T08:26:54.442 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.442 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.442 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.442 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.443 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.444 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.445 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.445 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.445 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.445 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:54 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T08:26:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:26:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:26:54.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-09-16T08:26:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-09-16T08:26:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-09-16T08:26:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-09-16T08:26:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: pgmap v16: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 27 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]: dispatch 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.y"}]': finished 2024-09-16T08:26:54.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:54.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:54.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:54.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:54.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:54.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:54 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:55.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: Upgrade: Setting container_image for all mgr 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: Health check update: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:55 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:55.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: Upgrade: Setting container_image for all mgr 2024-09-16T08:26:55.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:26:55.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:26:55.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: Upgrade: Setting container_image for all mgr 2024-09-16T08:26:55.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v17: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-16T08:26:55.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v18: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v19: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v20: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v21: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v22: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v23: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v24: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v25: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: pgmap v26: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: Health check update: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: Health check update: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:55.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:55 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:56 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v27: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v28: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v29: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v30: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v31: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v32: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v33: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v34: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v35: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v36: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v37: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v38: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v39: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v40: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v41: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v42: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v43: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v44: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v45: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v46: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v47: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: pgmap v48: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:56.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:26:56.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:56 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:26:57.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:57 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:57.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:57.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v49: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v50: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v51: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v52: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v53: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v54: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v55: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v56: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v57: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v58: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v59: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v60: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v61: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v62: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:26:57.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v63: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v64: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v65: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: pgmap v66: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:26:57.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:57 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:26:59.709 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:59 smithi193 ceph-mon[25614]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 2 op/s 2024-09-16T08:26:59.709 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:59 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:59.709 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:59 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:59.710 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:59 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:26:59.710 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:26:59 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:59.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[30018]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 2 op/s 2024-09-16T08:26:59.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:59.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:59.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:26:59.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:26:59.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[23701]: pgmap v67: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 2 op/s 2024-09-16T08:26:59.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:59.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:26:59.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:26:59.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:00.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:26:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:26:59] "GET /metrics HTTP/1.1" 200 38650 "" "Prometheus/2.43.0" 2024-09-16T08:27:00.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:00 smithi090 ceph-mon[30018]: Upgrade: Updating mgr.x 2024-09-16T08:27:00.565 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:00 smithi090 ceph-mon[30018]: Deploying daemon mgr.x on smithi193 2024-09-16T08:27:00.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:00 smithi090 ceph-mon[23701]: Upgrade: Updating mgr.x 2024-09-16T08:27:00.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:00 smithi090 ceph-mon[23701]: Deploying daemon mgr.x on smithi193 2024-09-16T08:27:00.648 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:00 smithi193 ceph-mon[25614]: Upgrade: Updating mgr.x 2024-09-16T08:27:00.648 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:00 smithi193 ceph-mon[25614]: Deploying daemon mgr.x on smithi193 2024-09-16T08:27:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:01 smithi193 ceph-mon[25614]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:27:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:01 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:01 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:01.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:01 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:01.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:01 smithi090 ceph-mon[30018]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:27:01.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:01 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:01.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:01 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:01.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:01 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:01.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:01 smithi090 ceph-mon[23701]: pgmap v68: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:27:01.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:01 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:01.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:01 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:01.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:01 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:02.565 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:02 smithi193 ceph-mon[25614]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:27:02.565 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:02 smithi193 ceph-mon[25614]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:27:02.565 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:02 smithi193 ceph-mon[25614]: Detected new or changed devices on smithi193 2024-09-16T08:27:02.565 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:02 smithi193 ceph-mon[25614]: Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:27:02.698 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:02 smithi090 ceph-mon[30018]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:27:02.698 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:02 smithi090 ceph-mon[30018]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:27:02.698 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:02 smithi090 ceph-mon[30018]: Detected new or changed devices on smithi193 2024-09-16T08:27:02.698 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:02 smithi090 ceph-mon[30018]: Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:27:02.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:02 smithi090 ceph-mon[23701]: pgmap v69: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:27:02.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:02 smithi090 ceph-mon[23701]: pgmap v70: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T08:27:02.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:02 smithi090 ceph-mon[23701]: Detected new or changed devices on smithi193 2024-09-16T08:27:02.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:02 smithi090 ceph-mon[23701]: Health check update: 2 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:27:03.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:03 smithi193 systemd[1]: Stopping Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:04.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:04 smithi193 ceph-mon[25614]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T08:27:04.502 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:04 smithi193 ceph-mon[25614]: from='client.24997 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:27:04.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:04 smithi090 ceph-mon[23701]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T08:27:04.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:04 smithi090 ceph-mon[23701]: from='client.24997 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:27:04.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:04 smithi090 ceph-mon[30018]: pgmap v71: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T08:27:04.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:04 smithi090 ceph-mon[30018]: from='client.24997 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:27:05.080 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:04 smithi193 podman[81633]: 2024-09-16 08:27:04.785734692 +0000 UTC m=+0.454975729 container died 23e7ac0df0bffa3e17b0cd1d47c57447e79249b6d6364cef441890f1d435696e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T08:27:05.351 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:05 smithi193 podman[81633]: 2024-09-16 08:27:05.27106541 +0000 UTC m=+0.940306447 container cleanup 23e7ac0df0bffa3e17b0cd1d47c57447e79249b6d6364cef441890f1d435696e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:27:05.352 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:05 smithi193 bash[81633]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x 2024-09-16T08:27:05.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[30018]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:27:05.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[30018]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 812 B/s rd, 0 op/s 2024-09-16T08:27:05.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[23701]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:27:05.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[23701]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 812 B/s rd, 0 op/s 2024-09-16T08:27:05.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:05 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.611 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:05 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.611 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:05 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:05 smithi193 ceph-mon[25614]: pgmap v72: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:27:05.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:05 smithi193 ceph-mon[25614]: pgmap v73: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 812 B/s rd, 0 op/s 2024-09-16T08:27:05.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:05 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.612 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:05 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:05.612 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:05 smithi193 podman[81727]: 2024-09-16 08:27:05.552923401 +0000 UTC m=+0.765384609 container remove 23e7ac0df0bffa3e17b0cd1d47c57447e79249b6d6364cef441890f1d435696e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T08:27:05.612 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:05 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-16T08:27:05.930 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:05 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-16T08:27:05.930 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:05 smithi193 systemd[1]: Stopped Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:27:05.930 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:05 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Consumed 20.667s CPU time. 2024-09-16T08:27:06.224 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:05 smithi193 systemd[1]: Starting Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:06.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:06 smithi090 ceph-mon[23701]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 812 B/s rd, 0 op/s 2024-09-16T08:27:06.612 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:06 smithi090 ceph-mon[30018]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 812 B/s rd, 0 op/s 2024-09-16T08:27:06.875 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:06 smithi193 ceph-mon[25614]: pgmap v74: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 812 B/s rd, 0 op/s 2024-09-16T08:27:06.876 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:06 smithi193 podman[82033]: 2024-09-16 08:27:06.777772882 +0000 UTC m=+0.113078120 container create ccce5e7a031428914ad49ffca87d7c21ed936a070750ff4b95d82d0880884c2a (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T08:27:06.876 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:06 smithi193 podman[82033]: 2024-09-16 08:27:06.682626334 +0000 UTC m=+0.017931593 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:27:07.184 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:06 smithi193 podman[82033]: 2024-09-16 08:27:06.980279382 +0000 UTC m=+0.315584622 container init ccce5e7a031428914ad49ffca87d7c21ed936a070750ff4b95d82d0880884c2a (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, io.buildah.version=1.37.2) 2024-09-16T08:27:07.184 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:06 smithi193 podman[82033]: 2024-09-16 08:27:06.983457868 +0000 UTC m=+0.318763105 container start ccce5e7a031428914ad49ffca87d7c21ed936a070750ff4b95d82d0880884c2a (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T08:27:07.185 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mgr[82065]: -- 172.21.15.193:0/2961460988 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55af1056cd00 con 0x55af10541000 2024-09-16T08:27:07.185 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:07 smithi193 bash[82033]: ccce5e7a031428914ad49ffca87d7c21ed936a070750ff4b95d82d0880884c2a 2024-09-16T08:27:07.185 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:07 smithi193 systemd[1]: Started Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:27:07.185 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:07.107+0000 7fb8a7c92040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T08:27:07.485 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:07.183+0000 7fb8a7c92040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T08:27:07.485 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:07.352+0000 7fb8a7c92040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T08:27:07.485 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:07.423+0000 7fb8a7c92040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T08:27:07.751 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:07.578+0000 7fb8a7c92040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:27:07.751 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:07.724+0000 7fb8a7c92040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: Health check update: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:07 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: Health check update: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:27:07.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:07.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:07.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: Health check update: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:27:07.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: pgmap v75: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: pgmap v76: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 784 B/s rd, 0 op/s 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: pgmap v77: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: pgmap v78: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: pgmap v79: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: pgmap v80: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:07.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:07.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:07.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:07.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: pgmap v81: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:07.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: pgmap v82: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:07.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:07.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:07 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.334 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:08.078+0000 7fb8a7c92040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T08:27:08.334 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:08.141+0000 7fb8a7c92040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T08:27:08.334 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:08.333+0000 7fb8a7c92040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T08:27:08.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:08.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:08.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 17 op/s 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 17 op/s 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.764 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.765 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.766 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:08 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 17 op/s 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 17 op/s 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v83: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v84: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v85: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 17 op/s 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v86: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 17 op/s 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v87: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v88: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v89: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v90: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v91: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v92: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v93: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v94: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v95: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v96: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v97: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v98: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v99: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v100: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v101: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v102: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:08.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:08.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:08.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:08.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:08.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v103: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: pgmap v104: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:08.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:08.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:08 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.493 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:09.252+0000 7fb8a7c92040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T08:27:09.493 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:09.323+0000 7fb8a7c92040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T08:27:09.812 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:27:09] "GET /metrics HTTP/1.1" 200 38650 "" "Prometheus/2.43.0" 2024-09-16T08:27:09.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:09 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.951 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.951 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v105: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v106: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.952 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.953 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.953 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v107: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.953 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v108: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.953 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.953 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.953 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.953 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v109: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v110: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.954 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v111: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v112: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v113: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v114: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.955 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v115: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v116: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.956 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.957 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.957 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v117: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.957 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v118: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.957 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.957 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.957 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v119: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v120: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.958 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v121: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v122: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v123: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.959 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v124: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v125: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: pgmap v126: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.960 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:09.961 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:09.961 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:09.961 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:09.961 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:09 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.203 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:10.083+0000 7fb8a7c92040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T08:27:10.469 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:10.203+0000 7fb8a7c92040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T08:27:10.470 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:10.354+0000 7fb8a7c92040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T08:27:10.470 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:10.430+0000 7fb8a7c92040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T08:27:10.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:10.584+0000 7fb8a7c92040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T08:27:10.753 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:10.655+0000 7fb8a7c92040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T08:27:10.753 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:10.716+0000 7fb8a7c92040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T08:27:10.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v127: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v128: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v129: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v130: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v131: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v132: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v133: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v134: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.821 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.822 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v135: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v136: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v137: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v138: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v139: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v140: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v141: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v142: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v143: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v144: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v145: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v146: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v147: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: pgmap v148: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:10 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:10.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch ps' 2024-09-16T08:27:11.079 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:27:11.146 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:10.879+0000 7fb8a7c92040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T08:27:11.147 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:10.952+0000 7fb8a7c92040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T08:27:11.147 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:11.020+0000 7fb8a7c92040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T08:27:11.147 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:11.105+0000 7fb8a7c92040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T08:27:11.398 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:11.168+0000 7fb8a7c92040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T08:27:11.398 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:11.233+0000 7fb8a7c92040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T08:27:11.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:11 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v149: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v150: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v151: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v152: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v153: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v154: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v155: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v156: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v157: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v158: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v159: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v160: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v161: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v162: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v163: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v164: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v165: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v166: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v167: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v168: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:11.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:11.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:11.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:11.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v169: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: pgmap v170: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:11.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:11.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:11 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.377 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:12.077+0000 7fb8a7c92040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T08:27:12.377 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: [16/Sep/2024:08:27:12] ENGINE Bus STARTING 2024-09-16T08:27:12.377 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: CherryPy Checker: 2024-09-16T08:27:12.377 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: The Application mounted at '' has an empty config. 2024-09-16T08:27:12.377 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: 2024-09-16T08:27:12.377 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: [16/Sep/2024:08:27:12] ENGINE Serving on http://:::9283 2024-09-16T08:27:12.377 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[82061]: [16/Sep/2024:08:27:12] ENGINE Bus STARTED 2024-09-16T08:27:12.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:27:12.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.668 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.668 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.669 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.670 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.671 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.672 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.673 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.673 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.673 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.673 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.673 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.673 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.673 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.673 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.673 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Standby manager daemon x restarted 2024-09-16T08:27:12.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Standby manager daemon x started 2024-09-16T08:27:12.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:27:12.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:27:12.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Standby manager daemon x restarted 2024-09-16T08:27:12.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Standby manager daemon x started 2024-09-16T08:27:12.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:27:12.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:27:12.678 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:27:12.679 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:27:12.679 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.679 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.679 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.679 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.680 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.680 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.680 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.680 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.680 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.681 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.681 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.681 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.681 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.681 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.681 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.682 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.682 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.682 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.682 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.682 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.682 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.683 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.683 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.683 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.683 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.683 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.683 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.683 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.684 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.684 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.684 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.684 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.684 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:12.685 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:12.685 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.685 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.685 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.685 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.685 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.686 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.686 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.686 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:27:12.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:27:12.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:12.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:12.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:12 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v171: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v172: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.754 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v173: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v174: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.755 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:27:12.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.756 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v175: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v176: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.757 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v177: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v178: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.758 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v179: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v180: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.759 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v181: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v182: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Standby manager daemon x restarted 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Standby manager daemon x started 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:27:12.760 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.? 172.21.15.193:0/917591680' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v183: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v184: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.761 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v185: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v186: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v187: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v188: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.762 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v189: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: pgmap v190: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:12.763 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:12 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.106 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:27:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi090 smithi090 running 8s ago 11m - - 2024-09-16T08:27:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:agent.smithi193 smithi193 running 12s ago 11m - - 2024-09-16T08:27:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.a smithi090 *:9093,9094 running (6m) 8s ago 7m 26.8M - 0.25.0 c8568f914cd2 3c66ea902fdc 2024-09-16T08:27:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.a smithi193 *:3000 running (6m) 12s ago 6m 87.1M - 9.4.7 954c08fa6188 d2a25f536ef6 2024-09-16T08:27:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:iscsi.foo.smithi090.jxkvvk smithi090 *:5000 running (21s) 8s ago 5m 50.3M - 3.8 fd1d6ca2b4a5 ee27cf31239d 2024-09-16T08:27:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi193 *:8443,9283,8765 starting - - - - 2024-09-16T08:27:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi090 *:8443,9283,8765 running (38s) 8s ago 12m 511M - 19.3.0-4917-g8293d73f 425e316ba8eb fa011575ebee 2024-09-16T08:27:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (12m) 8s ago 12m 56.7M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 87c2b7a329d6 2024-09-16T08:27:13.107 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi193 running (11m) 12s ago 11m 45.2M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a45dcb4d7793 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi090 running (11m) 8s ago 11m 44.5M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 fddfc4718a9c 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.a smithi090 *:9100 running (6m) 8s ago 6m 16.2M - 1.5.0 0da6a335fe13 4a6ade533b17 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.b smithi193 *:9100 running (6m) 12s ago 6m 16.0M - 1.5.0 0da6a335fe13 46a72e78b0a2 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (10m) 8s ago 10m 53.9M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 d8ac704cb999 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (10m) 8s ago 10m 60.0M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0703646c72eb 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (9m) 8s ago 9m 49.2M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3f0553bc3456 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (9m) 8s ago 9m 60.2M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90097820090a 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi193 running (9m) 12s ago 9m 53.4M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 389bfe865f47 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi193 running (8m) 12s ago 8m 50.9M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 b5fa3f6ab4bf 2024-09-16T08:27:13.108 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi193 running (8m) 12s ago 8m 61.4M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 dc9317ffcb0d 2024-09-16T08:27:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi193 running (7m) 12s ago 7m 61.9M 4096M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 9a7223ca10b1 2024-09-16T08:27:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.a smithi193 *:9095 unknown 12s ago 7m 36.5M - 2.43.0 a07b618ecd1d 54a1e828f158 2024-09-16T08:27:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi090.fbrjws smithi090 *:8000 running (5m) 8s ago 5m 78.7M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 6bd392324db2 2024-09-16T08:27:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.foo.smithi193.gjyvkw smithi193 *:8000 running (5m) 12s ago 5m 77.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 351896279a6f 2024-09-16T08:27:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi090.ebelfk smithi090 *:80 running (5m) 8s ago 5m 77.9M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 374e9c0e942b 2024-09-16T08:27:13.109 INFO:teuthology.orchestra.run.smithi090.stdout:rgw.smpl.smithi193.cgoeeh smithi193 *:80 running (5m) 12s ago 5m 78.7M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cf9d24a65b38 2024-09-16T08:27:13.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:13.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:13.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.812 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.813 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='client.24997 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='client.24997 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:27:13.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.818 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: mgrmap e32: y(active, since 31s), standbys: x 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.819 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: mgrmap e32: y(active, since 31s), standbys: x 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:27:13.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.826 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.827 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.830 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:13 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v191: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:13.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v192: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 18 op/s 2024-09-16T08:27:13.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v193: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v194: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 18 op/s 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v195: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v196: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='client.24997 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v197: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v198: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v199: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v200: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v201: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v202: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: mgrmap e32: y(active, since 31s), standbys: x 2024-09-16T08:27:13.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v203: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v204: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v205: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v206: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v207: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v208: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.839 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v209: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v210: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.840 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v211: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: pgmap v212: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:13.841 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.842 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:13.842 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:13.842 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:13.842 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:13.842 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:13 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:13.846 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph versions' 2024-09-16T08:27:14.061 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:27:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.814 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.815 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.816 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.817 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v220: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v221: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v222: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v223: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v224: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.819 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v225: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v226: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v227: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v228: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v229: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v230: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v231: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v232: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v233: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: pgmap v234: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.823 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v220: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v221: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v222: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.824 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v223: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v224: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.825 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v225: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v226: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v227: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.826 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v228: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v229: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v230: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.827 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v231: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v232: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.828 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v233: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: pgmap v234: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.829 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:14 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.830 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.830 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v213: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.830 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v214: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.830 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.830 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.830 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.830 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.830 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v215: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v216: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.831 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v217: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v218: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v219: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v220: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.832 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v221: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v222: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.833 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v223: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v224: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.834 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v225: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v226: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v227: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v228: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.835 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v229: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v230: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.836 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v231: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v232: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.837 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v233: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: pgmap v234: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:14.838 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:14 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.685 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.685 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v235: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.685 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v236: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.685 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v237: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v238: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.686 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v239: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v240: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.687 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v241: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v242: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.688 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v243: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v244: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v245: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v246: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.689 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.690 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.690 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.690 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.690 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.690 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.690 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.690 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v247: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v248: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.691 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v250: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v251: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v252: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.692 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v253: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: pgmap v254: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.693 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:15 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.781 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v235: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v236: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v237: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v238: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.782 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v239: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v240: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.783 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.784 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v235: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v236: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v237: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v238: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v239: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v240: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v241: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v242: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v243: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v244: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v245: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v246: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.788 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.789 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.789 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v241: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.789 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v242: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.789 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.789 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.789 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.789 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.789 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v243: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v244: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.790 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v245: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v246: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v247: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v248: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.791 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v250: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.792 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v251: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v252: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v253: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.793 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: pgmap v254: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v247: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v248: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v249: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v250: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v251: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v252: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v253: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: pgmap v254: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:15.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:15 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:16.245 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:27:16.245 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-09-16T08:27:16.245 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 3 2024-09-16T08:27:16.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:27:16.245 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-09-16T08:27:16.245 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 1, 2024-09-16T08:27:16.245 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 8 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: "rgw": { 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 16, 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-16T08:27:16.246 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:27:17.044 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch upgrade status' 2024-09-16T08:27:17.286 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:27:17.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:17 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/3049089829' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:17.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:17 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/3049089829' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:17.421 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:17 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/3049089829' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:18.624 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[30018]: pgmap v255: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:18.624 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[30018]: Upgrade: Updating mgr.x 2024-09-16T08:27:18.624 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:18.624 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:27:18.624 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:27:18.624 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:27:18.624 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:18.624 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[30018]: Deploying daemon mgr.x on smithi193 2024-09-16T08:27:18.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[23701]: pgmap v255: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[23701]: Upgrade: Updating mgr.x 2024-09-16T08:27:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:27:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:27:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:27:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:18 smithi090 ceph-mon[23701]: Deploying daemon mgr.x on smithi193 2024-09-16T08:27:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:18 smithi193 ceph-mon[25614]: pgmap v255: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail 2024-09-16T08:27:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:18 smithi193 ceph-mon[25614]: Upgrade: Updating mgr.x 2024-09-16T08:27:18.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:18 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:18.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:18 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:27:18.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:18 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:27:18.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:18 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:27:18.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:18 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:18.753 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:18 smithi193 ceph-mon[25614]: Deploying daemon mgr.x on smithi193 2024-09-16T08:27:19.022 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-16T08:27:19.022 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T08:27:19.022 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-09-16T08:27:19.023 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:27:19.023 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-09-16T08:27:19.023 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "", 2024-09-16T08:27:19.023 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T08:27:19.023 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-09-16T08:27:19.023 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-16T08:27:19.706 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph health detail' 2024-09-16T08:27:19.908 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:27:20.017 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:19 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:27:19] "GET /metrics HTTP/1.1" 200 38655 "" "Prometheus/2.43.0" 2024-09-16T08:27:20.191 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:19 smithi193 systemd[1]: Stopping Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:20.626 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:20 smithi090 ceph-mon[30018]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:27:20.627 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:20 smithi090 ceph-mon[30018]: pgmap v256: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:27:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:20 smithi090 ceph-mon[23701]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:27:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:20 smithi090 ceph-mon[23701]: pgmap v256: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:27:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:20 smithi193 ceph-mon[25614]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:27:20.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:20 smithi193 ceph-mon[25614]: pgmap v256: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T08:27:20.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:20 smithi193 podman[84702]: 2024-09-16 08:27:20.595673358 +0000 UTC m=+0.125427606 container died ccce5e7a031428914ad49ffca87d7c21ed936a070750ff4b95d82d0880884c2a (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, RELEASE=reef-ae7f373, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:27:21.224 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:20 smithi193 podman[84702]: 2024-09-16 08:27:20.964696519 +0000 UTC m=+0.494450766 container cleanup ccce5e7a031428914ad49ffca87d7c21ed936a070750ff4b95d82d0880884c2a (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-ae7f373, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T08:27:21.224 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:20 smithi193 bash[84702]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x 2024-09-16T08:27:21.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:21 smithi193 podman[84732]: 2024-09-16 08:27:21.224330408 +0000 UTC m=+0.628436826 container remove ccce5e7a031428914ad49ffca87d7c21ed936a070750ff4b95d82d0880884c2a (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T08:27:21.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:21 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-16T08:27:21.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:21 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-16T08:27:21.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:21 smithi193 systemd[1]: Stopped Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:27:21.502 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:21 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Consumed 5.975s CPU time. 2024-09-16T08:27:21.692 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 failed cephadm daemon(s) 2024-09-16T08:27:21.692 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-09-16T08:27:21.692 INFO:teuthology.orchestra.run.smithi090.stdout: daemon mgr.x on smithi193 is in unknown state 2024-09-16T08:27:21.879 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:21 smithi193 systemd[1]: Starting Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:21.993 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:21.993 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:21.993 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[30018]: pgmap v257: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 901 B/s rd, 0 op/s 2024-09-16T08:27:21.993 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[30018]: pgmap v258: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 901 B/s rd, 0 op/s 2024-09-16T08:27:21.993 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:21.993 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/1259939814' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:27:21.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:21.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:21.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[23701]: pgmap v257: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 901 B/s rd, 0 op/s 2024-09-16T08:27:21.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[23701]: pgmap v258: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 901 B/s rd, 0 op/s 2024-09-16T08:27:21.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:21.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:21 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/1259939814' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:27:22.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:21 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:22.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:21 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:22.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:21 smithi193 ceph-mon[25614]: pgmap v257: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 901 B/s rd, 0 op/s 2024-09-16T08:27:22.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:21 smithi193 ceph-mon[25614]: pgmap v258: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 901 B/s rd, 0 op/s 2024-09-16T08:27:22.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:21 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:22.252 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:21 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/1259939814' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:27:22.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-09-16T08:27:22.518 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T08:27:22.538 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:22 smithi193 podman[85031]: 2024-09-16 08:27:22.396567273 +0000 UTC m=+0.023535336 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T08:27:22.996 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:22 smithi193 podman[85031]: 2024-09-16 08:27:22.538870188 +0000 UTC m=+0.165838281 container create 7c382dc4e53265027c0ec41271c17db7e5add7e3bb907013d4229e4d0f4876a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T08:27:23.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:22 smithi193 podman[85031]: 2024-09-16 08:27:22.996347701 +0000 UTC m=+0.623315751 container init 7c382dc4e53265027c0ec41271c17db7e5add7e3bb907013d4229e4d0f4876a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:27:23.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:22 smithi193 podman[85031]: 2024-09-16 08:27:22.999581403 +0000 UTC m=+0.626549461 container start 7c382dc4e53265027c0ec41271c17db7e5add7e3bb907013d4229e4d0f4876a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T08:27:23.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:23 smithi193 ceph-mgr[85078]: -- 172.21.15.193:0/4099880406 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555cc28a10e0 con 0x555cc287e800 2024-09-16T08:27:23.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:23 smithi193 bash[85031]: 7c382dc4e53265027c0ec41271c17db7e5add7e3bb907013d4229e4d0f4876a6 2024-09-16T08:27:23.252 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:23 smithi193 systemd[1]: Started Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:27:24.438 INFO:teuthology.orchestra.run.smithi090.stdout:false 2024-09-16T08:27:24.439 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='client.24997 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:27:24.439 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: pgmap v259: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:27:24.439 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.440 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.440 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.440 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.440 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.440 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: pgmap v260: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:27:24.441 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.441 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.441 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.441 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.441 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:24.442 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:27:24.442 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:27:24.442 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-09-16T08:27:24.442 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.442 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.443 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='client.24997 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:27:24.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: pgmap v259: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:27:24.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: pgmap v260: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:27:24.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.446 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.446 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.446 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:24.446 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.446 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.446 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.446 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.446 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.446 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.447 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.448 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.449 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.449 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.449 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:24.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.452 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.452 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.452 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.452 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.452 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.452 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.452 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.453 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.454 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:24 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.503 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='client.24997 -' entity='client.iscsi.foo.smithi090.jxkvvk' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T08:27:24.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: pgmap v259: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:27:24.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.504 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: pgmap v260: 161 pgs: 161 active+clean; 585 KiB data, 254 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T08:27:24.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.505 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:24.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:27:24.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]: dispatch 2024-09-16T08:27:24.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.x"}]': finished 2024-09-16T08:27:24.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.506 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.507 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:24.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.508 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.509 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.510 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.511 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.512 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:24.513 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:24 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:27:25.047 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa/tasks/cephadm.py", line 1525, in shell _shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa/tasks/cephadm.py", line 111, in _shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi090 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2024-09-16T08:27:25.314 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=482547eb6dee47b48c8bbd4b6e9cf9f5 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa/tasks/cephadm.py", line 1525, in shell _shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa/tasks/cephadm.py", line 111, in _shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi090 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2024-09-16T08:27:25.317 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T08:27:25.328 INFO:tasks.cephadm:Teardown begin 2024-09-16T08:27:25.328 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:27:25.333 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.333 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.333 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.333 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.333 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.333 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-16T08:27:25.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Cluster is now healthy 2024-09-16T08:27:25.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.337 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.337 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.337 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.337 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.337 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.338 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.338 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.338 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.338 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.338 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.338 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.338 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.338 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.338 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.339 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.339 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.339 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-16T08:27:25.339 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Cluster is now healthy 2024-09-16T08:27:25.339 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.339 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.339 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.340 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.340 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.340 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.340 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.340 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.340 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.340 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.340 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.341 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.341 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.341 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.341 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.341 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.341 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.341 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.341 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.342 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.342 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.342 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.342 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.342 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.342 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='client.? 172.21.15.90:0/2994128477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.342 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.342 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.343 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.343 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.343 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.343 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.343 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.343 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.343 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.343 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.343 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.344 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:25 smithi193 ceph-mon[25614]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.344 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.344 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.344 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.344 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.344 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.344 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.345 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.345 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.345 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.345 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.345 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.345 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.345 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.346 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.346 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-16T08:27:25.346 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Cluster is now healthy 2024-09-16T08:27:25.346 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.346 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.346 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.346 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.346 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.347 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.348 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.348 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.348 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.348 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.348 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='client.? 172.21.15.90:0/2994128477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[23701]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.351 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.351 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.352 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.352 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.352 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.352 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.352 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.352 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.352 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='client.? 172.21.15.90:0/2994128477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.353 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.353 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:27:25.353 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Upgrade: 1 mgr daemon(s) are 18.2.4-853-gae7f3733 != target 19.3.0-4917-g8293d73f 2024-09-16T08:27:25.353 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.353 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:27:25.353 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:27:25.353 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.353 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-16T08:27:25.354 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.354 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.354 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 ' entity='mgr.y' 2024-09-16T08:27:25.354 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-mon[30018]: from='mgr.15339 172.21.15.90:0/1275894680' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:27:25.381 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:27:25.411 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T08:27:25.411 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T08:27:25.437 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T08:27:25.466 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T08:27:25.466 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T08:27:25.467 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.a 2024-09-16T08:27:25.755 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 systemd[1]: Stopping Ceph mon.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:26.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a[23697]: 2024-09-16T08:27:25.754+0000 7fb0083ca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:27:26.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a[23697]: 2024-09-16T08:27:25.754+0000 7fb0083ca640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T08:27:26.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:25 smithi090 podman[78384]: 2024-09-16 08:27:25.928567838 +0000 UTC m=+0.264468456 container died 87c2b7a329d6c2aae532fd9ba310cfa1379a04fe15e15c99df3f3e188e387f6d (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T08:27:26.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:26 smithi090 podman[78384]: 2024-09-16 08:27:26.280334179 +0000 UTC m=+0.616234796 container cleanup 87c2b7a329d6c2aae532fd9ba310cfa1379a04fe15e15c99df3f3e188e387f6d (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T08:27:26.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:26 smithi090 bash[78384]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-a 2024-09-16T08:27:26.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 16 08:27:26 smithi090 podman[78424]: 2024-09-16 08:27:26.735633935 +0000 UTC m=+0.433962425 container died 5107dcb31bea65d80297e597e062584f58678bc29b78179d5dab76c7a8b09733 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=jolly_swanson, RELEASE=reef-ae7f373, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True) 2024-09-16T08:27:27.143 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T08:27:27.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:27:27.192 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T08:27:27.193 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-16T08:27:27.193 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.c 2024-09-16T08:27:27.562 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:27 smithi090 systemd[1]: Stopping Ceph mon.c for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:28.720 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[85058]: [16/Sep/2024:08:27:28] ENGINE Bus STARTING 2024-09-16T08:27:28.720 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[85058]: CherryPy Checker: 2024-09-16T08:27:28.720 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[85058]: The Application mounted at '' has an empty config. 2024-09-16T08:27:28.720 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[85058]: 2024-09-16T08:27:28.721 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[85058]: [16/Sep/2024:08:27:28] ENGINE Serving on http://:::9283 2024-09-16T08:27:28.721 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x[85058]: [16/Sep/2024:08:27:28] ENGINE Bus STARTED 2024-09-16T08:27:28.794 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:28 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-c[30013]: 2024-09-16T08:27:28.508+0000 7f3237269640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:27:28.794 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:28 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-c[30013]: 2024-09-16T08:27:28.508+0000 7f3237269640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T08:27:28.794 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:28 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:27:28] ENGINE Bus STOPPING 2024-09-16T08:27:28.794 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:28 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:27:28] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T08:27:29.061 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:28 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:27:28] ENGINE Bus STOPPED 2024-09-16T08:27:29.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:28 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:27:28] ENGINE Bus STARTING 2024-09-16T08:27:29.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:28 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:27:28] ENGINE Serving on http://:::9283 2024-09-16T08:27:29.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:28 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: [16/Sep/2024:08:27:28] ENGINE Bus STARTED 2024-09-16T08:27:29.312 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:29 smithi090 podman[78592]: 2024-09-16 08:27:29.091712585 +0000 UTC m=+0.772279114 container died fddfc4718a9c86f0693dba0460b089bf2b02497db5026355aa94ecbc444bc13e (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T08:27:29.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:29 smithi090 podman[78592]: 2024-09-16 08:27:29.406100372 +0000 UTC m=+1.086666908 container cleanup fddfc4718a9c86f0693dba0460b089bf2b02497db5026355aa94ecbc444bc13e (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:27:29.674 INFO:journalctl@ceph.mon.c.smithi090.stdout:Sep 16 08:27:29 smithi090 bash[78592]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-c 2024-09-16T08:27:29.834 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T08:27:29.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:27:29.873 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-16T08:27:29.873 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-16T08:27:29.873 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.b 2024-09-16T08:27:30.062 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:29 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: ::ffff:172.21.15.193 - - [16/Sep/2024:08:27:29] "GET /metrics HTTP/1.1" 200 38653 "" "Prometheus/2.43.0" 2024-09-16T08:27:30.215 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:29 smithi193 systemd[1]: Stopping Ceph mon.b for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:31.101 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-b[25597]: 2024-09-16T08:27:30.797+0000 7f652f5c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:27:31.101 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-b[25597]: 2024-09-16T08:27:30.797+0000 7f652f5c6640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T08:27:31.473 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:31 smithi193 podman[86201]: 2024-09-16 08:27:31.101314819 +0000 UTC m=+0.400116072 container died a45dcb4d779321c56c725bacba8c37f1d03e625e1cf4586341daa2d0af9022c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux ) 2024-09-16T08:27:31.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:31 smithi193 podman[86201]: 2024-09-16 08:27:31.474230176 +0000 UTC m=+0.773031428 container cleanup a45dcb4d779321c56c725bacba8c37f1d03e625e1cf4586341daa2d0af9022c5 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-16T08:27:31.752 INFO:journalctl@ceph.mon.b.smithi193.stdout:Sep 16 08:27:31 smithi193 bash[86201]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mon-b 2024-09-16T08:27:32.077 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T08:27:32.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:27:32.134 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-16T08:27:32.134 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T08:27:32.134 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y 2024-09-16T08:27:32.486 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:32 smithi090 systemd[1]: Stopping Ceph mgr.y for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:32.812 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:32 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y[72253]: 2024-09-16T08:27:32.485+0000 7f427f3c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:27:32.812 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:32 smithi090 podman[78758]: 2024-09-16 08:27:32.527538202 +0000 UTC m=+0.196865562 container died fa011575ebeebc942d838bc1db10a329124aeb5da069379bd3a24250804a356a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2) 2024-09-16T08:27:33.198 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:32 smithi090 podman[78758]: 2024-09-16 08:27:32.92133858 +0000 UTC m=+0.590665930 container cleanup fa011575ebeebc942d838bc1db10a329124aeb5da069379bd3a24250804a356a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T08:27:33.198 INFO:journalctl@ceph.mgr.y.smithi090.stdout:Sep 16 08:27:32 smithi090 bash[78758]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-y 2024-09-16T08:27:33.346 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T08:27:33.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:27:33.382 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T08:27:33.382 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T08:27:33.382 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x 2024-09-16T08:27:33.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:33 smithi193 systemd[1]: Stopping Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:34.056 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:33 smithi193 podman[86608]: 2024-09-16 08:27:33.75237632 +0000 UTC m=+0.153865720 container died 7c382dc4e53265027c0ec41271c17db7e5add7e3bb907013d4229e4d0f4876a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T08:27:34.498 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:34 smithi193 podman[86608]: 2024-09-16 08:27:34.146435981 +0000 UTC m=+0.547925378 container cleanup 7c382dc4e53265027c0ec41271c17db7e5add7e3bb907013d4229e4d0f4876a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T08:27:34.498 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:34 smithi193 bash[86608]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x 2024-09-16T08:27:34.736 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T08:27:34.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:34 smithi193 podman[86670]: 2024-09-16 08:27:34.497889599 +0000 UTC m=+0.743439292 container remove 7c382dc4e53265027c0ec41271c17db7e5add7e3bb907013d4229e4d0f4876a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True) 2024-09-16T08:27:34.752 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:34 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-16T08:27:34.753 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:34 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-16T08:27:34.753 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:34 smithi193 systemd[1]: Stopped Ceph mgr.x for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:27:34.753 INFO:journalctl@ceph.mgr.x.smithi193.stdout:Sep 16 08:27:34 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@mgr.x.service: Consumed 6.384s CPU time. 2024-09-16T08:27:34.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:27:34.774 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T08:27:34.774 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T08:27:34.774 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.0 2024-09-16T08:27:35.062 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:34 smithi090 systemd[1]: Stopping Ceph osd.0 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:35.562 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:35 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[37906]: 2024-09-16T08:27:35.083+0000 7f4df29c7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:27:35.562 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:35 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[37906]: 2024-09-16T08:27:35.083+0000 7f4df29c7640 -1 osd.0 104 *** Got signal Terminated *** 2024-09-16T08:27:35.562 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:35 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0[37906]: 2024-09-16T08:27:35.083+0000 7f4df29c7640 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T08:27:40.845 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:40 smithi090 podman[78881]: 2024-09-16 08:27:40.576722726 +0000 UTC m=+5.610979701 container died d8ac704cb999ccfbdac7516e6ae4c8e9c4c405e9fc3ae47cd0b8eca4abe81f55 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:27:41.178 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:41 smithi090 podman[78881]: 2024-09-16 08:27:41.010456073 +0000 UTC m=+6.044713042 container cleanup d8ac704cb999ccfbdac7516e6ae4c8e9c4c405e9fc3ae47cd0b8eca4abe81f55 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T08:27:41.179 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:41 smithi090 bash[78881]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0 2024-09-16T08:27:41.562 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:41 smithi090 podman[79078]: 2024-09-16 08:27:41.178667516 +0000 UTC m=+0.601550134 container remove d8ac704cb999ccfbdac7516e6ae4c8e9c4c405e9fc3ae47cd0b8eca4abe81f55 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T08:27:42.009 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:41 smithi090 podman[79254]: 2024-09-16 08:27:41.56351266 +0000 UTC m=+0.017515342 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:27:42.009 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:41 smithi090 podman[79254]: 2024-09-16 08:27:41.67292377 +0000 UTC m=+0.126926484 container create b2998f17f4c16e2e16a8216ab342affd133144d86aa5733cb6bf226adb214743 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=reef-ae7f373) 2024-09-16T08:27:42.009 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:41 smithi090 podman[79254]: 2024-09-16 08:27:41.900897382 +0000 UTC m=+0.354900054 container init b2998f17f4c16e2e16a8216ab342affd133144d86aa5733cb6bf226adb214743 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T08:27:42.010 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:41 smithi090 podman[79254]: 2024-09-16 08:27:41.904309315 +0000 UTC m=+0.358311988 container start b2998f17f4c16e2e16a8216ab342affd133144d86aa5733cb6bf226adb214743 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T08:27:42.010 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:42 smithi090 podman[79254]: 2024-09-16 08:27:42.009139954 +0000 UTC m=+0.463142649 container attach b2998f17f4c16e2e16a8216ab342affd133144d86aa5733cb6bf226adb214743 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-ae7f373, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T08:27:42.312 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 16 08:27:42 smithi090 podman[79254]: 2024-09-16 08:27:42.091475002 +0000 UTC m=+0.545477675 container died b2998f17f4c16e2e16a8216ab342affd133144d86aa5733cb6bf226adb214743 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373) 2024-09-16T08:27:42.710 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T08:27:42.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:27:42.750 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T08:27:42.750 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T08:27:42.750 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.1 2024-09-16T08:27:42.991 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:42 smithi090 systemd[1]: Stopping Ceph osd.1 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:43.812 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1[38940]: 2024-09-16T08:27:43.554+0000 7f1adbec6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:27:43.812 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1[38940]: 2024-09-16T08:27:43.554+0000 7f1adbec6640 -1 osd.1 104 *** Got signal Terminated *** 2024-09-16T08:27:43.812 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:43 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1[38940]: 2024-09-16T08:27:43.554+0000 7f1adbec6640 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T08:27:49.312 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:49 smithi090 podman[79467]: 2024-09-16 08:27:49.139594999 +0000 UTC m=+5.706593896 container died 0703646c72eba6d605cfb8176cc50ead354f1d8d3be5d6245b25dc86c42278fc (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:27:49.662 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:49 smithi090 podman[79467]: 2024-09-16 08:27:49.475637776 +0000 UTC m=+6.042636664 container cleanup 0703646c72eba6d605cfb8176cc50ead354f1d8d3be5d6245b25dc86c42278fc (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=reef-ae7f373, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:27:49.662 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:49 smithi090 bash[79467]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1 2024-09-16T08:27:50.039 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:49 smithi090 podman[79849]: 2024-09-16 08:27:49.662145686 +0000 UTC m=+0.604318677 container remove 0703646c72eba6d605cfb8176cc50ead354f1d8d3be5d6245b25dc86c42278fc (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T08:27:50.812 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:50 smithi090 podman[79941]: 2024-09-16 08:27:50.436870196 +0000 UTC m=+0.015676651 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:27:50.814 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:50 smithi090 podman[79941]: 2024-09-16 08:27:50.541069016 +0000 UTC m=+0.119875459 container create e50dcaeb5c0513cfb9df060c1ff650cc0e8bfe95655f17147df4c8f5a5d9cd7f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=reef-ae7f373, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T08:27:50.814 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:50 smithi090 podman[79941]: 2024-09-16 08:27:50.760278848 +0000 UTC m=+0.339085292 container init e50dcaeb5c0513cfb9df060c1ff650cc0e8bfe95655f17147df4c8f5a5d9cd7f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373) 2024-09-16T08:27:50.814 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:50 smithi090 podman[79941]: 2024-09-16 08:27:50.763439615 +0000 UTC m=+0.342246052 container start e50dcaeb5c0513cfb9df060c1ff650cc0e8bfe95655f17147df4c8f5a5d9cd7f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T08:27:51.312 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:50 smithi090 podman[79941]: 2024-09-16 08:27:50.868943979 +0000 UTC m=+0.447750427 container attach e50dcaeb5c0513cfb9df060c1ff650cc0e8bfe95655f17147df4c8f5a5d9cd7f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T08:27:51.312 INFO:journalctl@ceph.osd.1.smithi090.stdout:Sep 16 08:27:50 smithi090 podman[79941]: 2024-09-16 08:27:50.955241346 +0000 UTC m=+0.534047783 container died e50dcaeb5c0513cfb9df060c1ff650cc0e8bfe95655f17147df4c8f5a5d9cd7f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=reef-ae7f373, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T08:27:51.571 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T08:27:51.611 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:27:51.612 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T08:27:51.612 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T08:27:51.612 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.2 2024-09-16T08:27:51.812 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:51 smithi090 systemd[1]: Stopping Ceph osd.2 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:27:52.312 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2[42433]: 2024-09-16T08:27:51.935+0000 7f93f856e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:27:52.312 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2[42433]: 2024-09-16T08:27:51.935+0000 7f93f856e640 -1 osd.2 104 *** Got signal Terminated *** 2024-09-16T08:27:52.313 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:51 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2[42433]: 2024-09-16T08:27:51.935+0000 7f93f856e640 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T08:27:54.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:27:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:27:53.922Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:27:54.253 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:27:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:27:53.922Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:27:54.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:27:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:27:53.922Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:27:54.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:27:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:27:53.925Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:27:54.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:27:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:27:53.925Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:27:54.254 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:27:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:27:53.929Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:27:54.255 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:27:53 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:27:53.929Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.90:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.90:8765: connect: connection refused" 2024-09-16T08:27:57.679 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:57 smithi090 podman[80044]: 2024-09-16 08:27:57.415475217 +0000 UTC m=+5.570895230 container died 3f0553bc3456e95dd33aa21e3f82a03597d35eb3358386e904bddea145de21ec (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T08:27:58.014 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:57 smithi090 podman[80044]: 2024-09-16 08:27:57.854799504 +0000 UTC m=+6.010219518 container cleanup 3f0553bc3456e95dd33aa21e3f82a03597d35eb3358386e904bddea145de21ec (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:27:58.014 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:57 smithi090 bash[80044]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2 2024-09-16T08:27:58.312 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:58 smithi090 podman[80059]: 2024-09-16 08:27:58.014646334 +0000 UTC m=+0.601760801 container remove 3f0553bc3456e95dd33aa21e3f82a03597d35eb3358386e904bddea145de21ec (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T08:27:58.735 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:58 smithi090 podman[80146]: 2024-09-16 08:27:58.407143702 +0000 UTC m=+0.104832718 container create cd6054e7939cc98cf33153fc46261b9a8b824fab6a1570c7c85e483be0163a4e (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:27:58.735 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:58 smithi090 podman[80146]: 2024-09-16 08:27:58.318423927 +0000 UTC m=+0.016112958 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:27:58.735 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:58 smithi090 podman[80146]: 2024-09-16 08:27:58.651895888 +0000 UTC m=+0.349584905 container init cd6054e7939cc98cf33153fc46261b9a8b824fab6a1570c7c85e483be0163a4e (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373) 2024-09-16T08:27:58.735 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:58 smithi090 podman[80146]: 2024-09-16 08:27:58.655080283 +0000 UTC m=+0.352769304 container start cd6054e7939cc98cf33153fc46261b9a8b824fab6a1570c7c85e483be0163a4e (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=reef-ae7f373) 2024-09-16T08:27:58.735 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:58 smithi090 podman[80146]: 2024-09-16 08:27:58.735199321 +0000 UTC m=+0.432888347 container attach cd6054e7939cc98cf33153fc46261b9a8b824fab6a1570c7c85e483be0163a4e (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=reef-ae7f373, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T08:27:59.062 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:58 smithi090 podman[80146]: 2024-09-16 08:27:58.844813505 +0000 UTC m=+0.542502533 container died cd6054e7939cc98cf33153fc46261b9a8b824fab6a1570c7c85e483be0163a4e (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:27:59.425 INFO:journalctl@ceph.osd.2.smithi090.stdout:Sep 16 08:27:59 smithi090 podman[80169]: 2024-09-16 08:27:59.415549864 +0000 UTC m=+0.565065074 container remove cd6054e7939cc98cf33153fc46261b9a8b824fab6a1570c7c85e483be0163a4e (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T08:27:59.430 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T08:27:59.468 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:27:59.469 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T08:27:59.469 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T08:27:59.469 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.3 2024-09-16T08:27:59.785 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:27:59 smithi090 systemd[1]: Stopping Ceph osd.3 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:28:00.062 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:27:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3[46095]: 2024-09-16T08:27:59.784+0000 7fd9d01e9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:28:00.062 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:27:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3[46095]: 2024-09-16T08:27:59.784+0000 7fd9d01e9640 -1 osd.3 104 *** Got signal Terminated *** 2024-09-16T08:28:00.062 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:27:59 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3[46095]: 2024-09-16T08:27:59.784+0000 7fd9d01e9640 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T08:28:01.752 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:01 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:01.499+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:02.752 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:02 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:02.508+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:02.811 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:02 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3[46095]: 2024-09-16T08:28:02.544+0000 7fd9c87f5640 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:35.851529+0000 front 2024-09-16T08:27:35.851638+0000 (oldest deadline 2024-09-16T08:28:01.751730+0000) 2024-09-16T08:28:03.431 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:03 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:03.209+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:03.752 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:03 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:03.537+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:03.812 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:03 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3[46095]: 2024-09-16T08:28:03.541+0000 7fd9c87f5640 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:35.851529+0000 front 2024-09-16T08:27:35.851638+0000 (oldest deadline 2024-09-16T08:28:01.751730+0000) 2024-09-16T08:28:04.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:04 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:04.222+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:04.817 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:04 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3[46095]: 2024-09-16T08:28:04.566+0000 7fd9c87f5640 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:35.851529+0000 front 2024-09-16T08:27:35.851638+0000 (oldest deadline 2024-09-16T08:28:01.751730+0000) 2024-09-16T08:28:05.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:04 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:04.581+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:05.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:05 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:05.223+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:05.549 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:05 smithi090 podman[80248]: 2024-09-16 08:28:05.278126278 +0000 UTC m=+5.586165418 container died 90097820090a1f0492958e5401b22a31af784488886ce6a2f7521c903356b90d (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731) 2024-09-16T08:28:05.812 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:05 smithi090 podman[80248]: 2024-09-16 08:28:05.630722305 +0000 UTC m=+5.938761484 container cleanup 90097820090a1f0492958e5401b22a31af784488886ce6a2f7521c903356b90d (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T08:28:05.812 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:05 smithi090 bash[80248]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3 2024-09-16T08:28:06.002 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:05 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:05.574+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:06.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:05 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:05.629+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:06.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:05 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:05.648+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:06.197 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:05 smithi090 podman[80751]: 2024-09-16 08:28:05.899857235 +0000 UTC m=+0.619813055 container remove 90097820090a1f0492958e5401b22a31af784488886ce6a2f7521c903356b90d (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T08:28:06.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:06 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:06.201+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:07.002 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:06 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:06.588+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:07.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:06 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:06.679+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:07.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:06 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:06.610+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:07.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:07.245+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:07.517 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:07 smithi090 podman[80981]: 2024-09-16 08:28:07.109553321 +0000 UTC m=+0.016972249 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:28:07.517 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:07 smithi090 podman[80981]: 2024-09-16 08:28:07.410796156 +0000 UTC m=+0.318215076 container create a0b1bdf461202d00340f694999917c8377fcc2960bee91c35da15132610f24a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T08:28:07.765 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:07.550+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:07.765 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:07.650+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:07.766 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:07 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:07.614+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:07.812 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:07 smithi090 podman[80981]: 2024-09-16 08:28:07.739217199 +0000 UTC m=+0.646636116 container init a0b1bdf461202d00340f694999917c8377fcc2960bee91c35da15132610f24a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=reef-ae7f373, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T08:28:07.812 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:07 smithi090 podman[80981]: 2024-09-16 08:28:07.743018207 +0000 UTC m=+0.650437127 container start a0b1bdf461202d00340f694999917c8377fcc2960bee91c35da15132610f24a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=reef-ae7f373, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:28:08.267 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:07 smithi090 podman[80981]: 2024-09-16 08:28:07.854063305 +0000 UTC m=+0.761482223 container attach a0b1bdf461202d00340f694999917c8377fcc2960bee91c35da15132610f24a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T08:28:08.600 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:08.278+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:09.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:08.647+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:09.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:08.601+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:09.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:08 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:08.600+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:09.403 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 16 08:28:09 smithi090 podman[80981]: 2024-09-16 08:28:09.157220314 +0000 UTC m=+2.064639234 container died a0b1bdf461202d00340f694999917c8377fcc2960bee91c35da15132610f24a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:28:09.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:09 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:09.239+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:09.682 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T08:28:09.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:28:09.722 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T08:28:09.722 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T08:28:09.722 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.4 2024-09-16T08:28:10.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:09 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:09.629+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:10.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:09 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:09.638+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:10.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:09 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:09.617+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:10.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:09 smithi193 systemd[1]: Stopping Ceph osd.4 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:28:10.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:10.243+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:10.502 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:10.040+0000 7f3e8157f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:28:10.502 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:10.040+0000 7f3e8157f640 -1 osd.4 104 *** Got signal Terminated *** 2024-09-16T08:28:10.503 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:10.040+0000 7f3e8157f640 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T08:28:11.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:10.584+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:11.002 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:10.571+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:11.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:10.662+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:11.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:10 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:10.662+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:11.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:11.238+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:12.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:11.555+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:12.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:11.556+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:12.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:11.682+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:12.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:11.682+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:12.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:11.611+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:12.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:11 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:11.611+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:45.311102+0000 front 2024-09-16T08:27:45.311167+0000 (oldest deadline 2024-09-16T08:28:11.211031+0000) 2024-09-16T08:28:12.592 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:12.276+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:12.593 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:12.276+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:13.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:12.591+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:13.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:12.591+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:13.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:12.703+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:13.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:12.703+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:13.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:12.610+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:13.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:12 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:12.610+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:45.311102+0000 front 2024-09-16T08:27:45.311167+0000 (oldest deadline 2024-09-16T08:28:11.211031+0000) 2024-09-16T08:28:13.592 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:13.306+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:13.592 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:13.306+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:14.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:13.591+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:14.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:13.591+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:14.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:13.675+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:14.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:13.675+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:14.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:13.606+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:14.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:13 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:13.606+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:45.311102+0000 front 2024-09-16T08:27:45.311167+0000 (oldest deadline 2024-09-16T08:28:11.211031+0000) 2024-09-16T08:28:14.629 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:14 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:14.344+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:14.629 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:14 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:14.344+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:15.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:14 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:14.627+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:15.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:14 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:14.627+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:15.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:14 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:14.640+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:15.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:14 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:14.640+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:15.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:14 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:14.653+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.510490+0000 front 2024-09-16T08:27:39.510430+0000 (oldest deadline 2024-09-16T08:28:04.810224+0000) 2024-09-16T08:28:15.004 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:14 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4[31975]: 2024-09-16T08:28:14.653+0000 7f3e7db93640 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:45.311102+0000 front 2024-09-16T08:27:45.311167+0000 (oldest deadline 2024-09-16T08:28:11.211031+0000) 2024-09-16T08:28:15.658 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:15 smithi193 podman[87715]: 2024-09-16 08:28:15.435024583 +0000 UTC m=+5.511320315 container died 389bfe865f470f523ac16fe33b8d7a2781a741ff128d8aaf1ec922fe7733bc51 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=reef-ae7f373, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T08:28:15.658 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:15 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:15.324+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:15.659 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:15 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:15.324+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:16.002 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:15 smithi193 podman[87715]: 2024-09-16 08:28:15.841150165 +0000 UTC m=+5.917445887 container cleanup 389bfe865f470f523ac16fe33b8d7a2781a741ff128d8aaf1ec922fe7733bc51 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T08:28:16.002 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:15 smithi193 bash[87715]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4 2024-09-16T08:28:16.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:15 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:15.666+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:16.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:15 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:15.666+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:16.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:15 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:15.664+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:16.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:15 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:15.664+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:16.404 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:16 smithi193 podman[87754]: 2024-09-16 08:28:16.084313442 +0000 UTC m=+0.642747453 container remove 389bfe865f470f523ac16fe33b8d7a2781a741ff128d8aaf1ec922fe7733bc51 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T08:28:16.405 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:16 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:16.298+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:16.405 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:16 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:16.298+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:16.678 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:16 smithi193 podman[87843]: 2024-09-16 08:28:16.307519622 +0000 UTC m=+0.016167212 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:28:16.678 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:16 smithi193 podman[87843]: 2024-09-16 08:28:16.410064554 +0000 UTC m=+0.118712131 container create 894ff0b6ef7aff339204c61f4e5f7790e985c64eb1c247ce57394dc7d0485239 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=reef-ae7f373) 2024-09-16T08:28:16.678 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:16 smithi193 podman[87843]: 2024-09-16 08:28:16.612378213 +0000 UTC m=+0.321025790 container init 894ff0b6ef7aff339204c61f4e5f7790e985c64eb1c247ce57394dc7d0485239 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T08:28:16.678 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:16 smithi193 podman[87843]: 2024-09-16 08:28:16.615618198 +0000 UTC m=+0.324265776 container start 894ff0b6ef7aff339204c61f4e5f7790e985c64eb1c247ce57394dc7d0485239 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T08:28:17.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:16 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:16.692+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:17.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:16 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:16.692+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:17.002 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:16 smithi193 podman[87843]: 2024-09-16 08:28:16.706846147 +0000 UTC m=+0.415493743 container attach 894ff0b6ef7aff339204c61f4e5f7790e985c64eb1c247ce57394dc7d0485239 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T08:28:17.003 INFO:journalctl@ceph.osd.4.smithi193.stdout:Sep 16 08:28:16 smithi193 podman[87843]: 2024-09-16 08:28:16.79721017 +0000 UTC m=+0.505857766 container died 894ff0b6ef7aff339204c61f4e5f7790e985c64eb1c247ce57394dc7d0485239 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-4-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T08:28:17.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:16 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:16.676+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:17.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:16 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:16.677+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:17.383 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:17.294+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:17.383 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:17.294+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:17.410 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T08:28:17.452 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:28:17.452 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T08:28:17.453 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T08:28:17.453 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.5 2024-09-16T08:28:17.646 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:17 smithi193 systemd[1]: Stopping Ceph osd.5 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:28:18.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:17.689+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:18.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:17.689+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:18.003 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:17.777+0000 7f6f35622640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:28:18.003 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:17.777+0000 7f6f35622640 -1 osd.5 104 *** Got signal Terminated *** 2024-09-16T08:28:18.003 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:17.777+0000 7f6f35622640 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T08:28:18.004 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:17.660+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:18.004 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:17.660+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:18.004 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:17 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:17.660+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:18.588 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:18 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:18.308+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:18.589 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:18 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:18.308+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:19.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:18 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:18.634+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:19.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:18 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:18.634+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:19.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:18 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:18.634+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:19.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:18 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:18.680+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:19.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:18 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:18.680+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:19.004 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:18 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:18.680+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:19.653 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:19.277+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:19.653 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:19.277+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:20.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:19.678+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:20.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:19.678+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:20.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:19.678+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:20.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:19.652+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:20.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:19.652+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:20.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:19 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:19.652+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:20.655 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:20 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:20.305+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:20.655 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:20 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:20.305+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:21.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:20 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:20.655+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:21.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:20 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:20.655+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:21.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:20 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:20.655+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:21.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:20 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:20.664+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:21.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:20 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:20.664+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:21.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:20 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:20.664+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:21.675 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:21.319+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:21.675 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:21.319+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:21.675 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:21.319+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:55.200481+0000 front 2024-09-16T08:27:55.200518+0000 (oldest deadline 2024-09-16T08:28:20.500258+0000) 2024-09-16T08:28:22.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:21.708+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:22.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:21.708+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:22.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:21.708+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:22.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:21.674+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:22.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:21.674+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:22.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:21 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:21.674+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:22.650 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:22.296+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:38.897921+0000 front 2024-09-16T08:27:38.897801+0000 (oldest deadline 2024-09-16T08:28:02.397760+0000) 2024-09-16T08:28:22.650 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:22.296+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:46.999243+0000 front 2024-09-16T08:27:46.999277+0000 (oldest deadline 2024-09-16T08:28:11.699244+0000) 2024-09-16T08:28:22.651 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5[35397]: 2024-09-16T08:28:22.296+0000 7f6f31c36640 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:55.200481+0000 front 2024-09-16T08:27:55.200518+0000 (oldest deadline 2024-09-16T08:28:20.500258+0000) 2024-09-16T08:28:22.903 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:22.649+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:22.903 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:22.649+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:22.903 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:22.649+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:22.904 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:22.695+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:22.904 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:22.695+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:22.904 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:22 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:22.695+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:23.502 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:23 smithi193 podman[87946]: 2024-09-16 08:28:23.168369941 +0000 UTC m=+5.482578149 container died b5fa3f6ab4bf8a05d665d80fd97874334b9d4f3faca87aa8cacec9f0adc0ef2c (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T08:28:23.868 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:23 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:23.609+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:23.868 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:23 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:23.609+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:23.868 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:23 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:23.609+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:23.869 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:23 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:23.740+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:23.869 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:23 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:23.740+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:23.869 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:23 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:23.740+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:23.869 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:23 smithi193 podman[87946]: 2024-09-16 08:28:23.564641654 +0000 UTC m=+5.878849871 container cleanup b5fa3f6ab4bf8a05d665d80fd97874334b9d4f3faca87aa8cacec9f0adc0ef2c (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T08:28:23.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:23 smithi193 bash[87946]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5 2024-09-16T08:28:23.870 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:23 smithi193 podman[88003]: 2024-09-16 08:28:23.732818408 +0000 UTC m=+0.643392508 container remove b5fa3f6ab4bf8a05d665d80fd97874334b9d4f3faca87aa8cacec9f0adc0ef2c (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=reef-ae7f373, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T08:28:24.900 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:24 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:24.559+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:24.900 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:24 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:24.559+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:24.901 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:24 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:24.559+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:24.901 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:24 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:24.725+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:24.901 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:24 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:24.725+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:24.901 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:24 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:24.725+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:25.252 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:24 smithi193 podman[88135]: 2024-09-16 08:28:24.803035611 +0000 UTC m=+0.017990442 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:28:25.252 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:24 smithi193 podman[88135]: 2024-09-16 08:28:24.913853721 +0000 UTC m=+0.128808578 container create 1f5bc0c428fed026a987127d9232787bcadfd82e43be8ea6e997d4f495ab3fa8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T08:28:25.897 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:25 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:25.606+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:25.897 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:25 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:25.606+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:25.897 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:25 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:25.606+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:25.898 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:25 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:25.693+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:25.898 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:25 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:25.693+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:25.898 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:25 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:25.693+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:25.898 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:25 smithi193 podman[88135]: 2024-09-16 08:28:25.711737489 +0000 UTC m=+0.926692305 container init 1f5bc0c428fed026a987127d9232787bcadfd82e43be8ea6e997d4f495ab3fa8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=reef-ae7f373, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T08:28:25.898 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:25 smithi193 podman[88135]: 2024-09-16 08:28:25.715147914 +0000 UTC m=+0.930102733 container start 1f5bc0c428fed026a987127d9232787bcadfd82e43be8ea6e997d4f495ab3fa8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:28:25.898 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:25 smithi193 podman[88135]: 2024-09-16 08:28:25.786971789 +0000 UTC m=+1.001926616 container attach 1f5bc0c428fed026a987127d9232787bcadfd82e43be8ea6e997d4f495ab3fa8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=reef-ae7f373, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T08:28:26.181 INFO:journalctl@ceph.osd.5.smithi193.stdout:Sep 16 08:28:25 smithi193 podman[88135]: 2024-09-16 08:28:25.897618568 +0000 UTC m=+1.112573392 container died 1f5bc0c428fed026a987127d9232787bcadfd82e43be8ea6e997d4f495ab3fa8 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T08:28:26.485 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T08:28:26.526 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:28:26.527 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T08:28:26.527 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T08:28:26.527 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.6 2024-09-16T08:28:26.738 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:26 smithi193 systemd[1]: Stopping Ceph osd.6 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:28:26.738 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:26.632+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:26.739 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:26.632+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:26.739 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:26.632+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:26.739 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:26.632+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:00.500151+0000 front 2024-09-16T08:28:00.500112+0000 (oldest deadline 2024-09-16T08:28:25.800073+0000) 2024-09-16T08:28:27.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:26.867+0000 7febc15e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:28:27.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:26.867+0000 7febc15e0640 -1 osd.6 104 *** Got signal Terminated *** 2024-09-16T08:28:27.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:26.867+0000 7febc15e0640 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T08:28:27.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:26.742+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:27.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:26.742+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:27.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:26.742+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:27.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:26 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:26.742+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:28.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:27.592+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:28.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:27.592+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:28.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:27.592+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:28.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:27.592+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:00.500151+0000 front 2024-09-16T08:28:00.500112+0000 (oldest deadline 2024-09-16T08:28:25.800073+0000) 2024-09-16T08:28:28.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:27.735+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:28.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:27.735+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:28.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:27.735+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:28.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:27 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:27.735+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:28.752 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:28.642+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:28.752 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:28.642+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:28.752 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:28.642+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:28.752 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:28.642+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:00.500151+0000 front 2024-09-16T08:28:00.500112+0000 (oldest deadline 2024-09-16T08:28:25.800073+0000) 2024-09-16T08:28:29.053 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:28.773+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:29.053 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:28.773+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:29.054 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:28.773+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:29.054 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:28 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:28.773+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:30.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:29 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:29.781+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:30.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:29 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:29.781+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:30.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:29 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:29.781+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:30.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:29 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:29.781+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:30.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:29 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:29.680+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:30.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:29 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:29.680+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:30.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:29 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:29.680+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:30.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:29 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:29.680+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:00.500151+0000 front 2024-09-16T08:28:00.500112+0000 (oldest deadline 2024-09-16T08:28:25.800073+0000) 2024-09-16T08:28:30.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:29 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:29.680+0000 7febb9bec640 -1 osd.6 104 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24734.0:916 4.1f 4:f95f44c2:::notify.0:head [watch ping cookie 94103741471488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) 2024-09-16T08:28:30.996 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:30.774+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:30.996 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:30.774+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:30.996 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:30.774+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:30.997 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:30.774+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:30.997 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:30.714+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:30.997 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:30.714+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:30.997 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:30.714+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:30.998 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:30.714+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:00.500151+0000 front 2024-09-16T08:28:00.500112+0000 (oldest deadline 2024-09-16T08:28:25.800073+0000) 2024-09-16T08:28:30.998 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:30 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:30.715+0000 7febb9bec640 -1 osd.6 104 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.24734.0:916 4.1f 4:f95f44c2:::notify.0:head [watch ping cookie 94103741471488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) 2024-09-16T08:28:32.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:31 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:31.700+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:39.398380+0000 front 2024-09-16T08:27:39.398218+0000 (oldest deadline 2024-09-16T08:28:04.698215+0000) 2024-09-16T08:28:32.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:31 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:31.700+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:44.698607+0000 front 2024-09-16T08:27:44.698596+0000 (oldest deadline 2024-09-16T08:28:10.598531+0000) 2024-09-16T08:28:32.002 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:31 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:31.700+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:53.499472+0000 front 2024-09-16T08:27:53.499517+0000 (oldest deadline 2024-09-16T08:28:17.599381+0000) 2024-09-16T08:28:32.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:31 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:31.700+0000 7febb9bec640 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:00.500151+0000 front 2024-09-16T08:28:00.500112+0000 (oldest deadline 2024-09-16T08:28:25.800073+0000) 2024-09-16T08:28:32.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:31 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6[39030]: 2024-09-16T08:28:31.701+0000 7febb9bec640 -1 osd.6 104 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.24734.0:916 4.1f 4:f95f44c2:::notify.0:head [watch ping cookie 94103741471488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e104) 2024-09-16T08:28:32.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:31 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:31.813+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:32.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:31 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:31.813+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:32.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:31 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:31.813+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:32.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:31 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:31.813+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:32.543 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:32 smithi193 podman[88309]: 2024-09-16 08:28:32.255738157 +0000 UTC m=+5.488633255 container died dc9317ffcb0d5daa9dcfa186f9ab6d7e0c545a9dd3e4822e03a12335d3c177ce (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=reef-ae7f373, CEPH_POINT_RELEASE=) 2024-09-16T08:28:32.861 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:32 smithi193 podman[88309]: 2024-09-16 08:28:32.717233093 +0000 UTC m=+5.950128234 container cleanup dc9317ffcb0d5daa9dcfa186f9ab6d7e0c545a9dd3e4822e03a12335d3c177ce (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux ) 2024-09-16T08:28:32.861 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:32 smithi193 bash[88309]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6 2024-09-16T08:28:33.252 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:32 smithi193 podman[88924]: 2024-09-16 08:28:32.993446942 +0000 UTC m=+0.731570303 container remove dc9317ffcb0d5daa9dcfa186f9ab6d7e0c545a9dd3e4822e03a12335d3c177ce (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:28:33.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:32 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:32.860+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:33.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:32 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:32.860+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:33.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:32 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:32.860+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:33.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:32 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:32.860+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:33.666 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:33 smithi193 podman[89015]: 2024-09-16 08:28:33.23406918 +0000 UTC m=+0.015905574 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:28:33.666 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:33 smithi193 podman[89015]: 2024-09-16 08:28:33.375513027 +0000 UTC m=+0.157349411 container create a26dd794dd843e576d215c07d4de3dc70e875fa79321d44755a6c9b11d93e60f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:28:34.002 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:33 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:33.842+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:34.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:33 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:33.842+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:34.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:33 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:33.842+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:34.003 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:33 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:33.842+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:34.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:33 smithi193 podman[89015]: 2024-09-16 08:28:33.666213877 +0000 UTC m=+0.448050267 container init a26dd794dd843e576d215c07d4de3dc70e875fa79321d44755a6c9b11d93e60f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T08:28:34.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:33 smithi193 podman[89015]: 2024-09-16 08:28:33.669792614 +0000 UTC m=+0.451629001 container start a26dd794dd843e576d215c07d4de3dc70e875fa79321d44755a6c9b11d93e60f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:28:34.003 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:33 smithi193 podman[89015]: 2024-09-16 08:28:33.821929657 +0000 UTC m=+0.603766055 container attach a26dd794dd843e576d215c07d4de3dc70e875fa79321d44755a6c9b11d93e60f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, org.label-schema.schema-version=1.0) 2024-09-16T08:28:34.004 INFO:journalctl@ceph.osd.6.smithi193.stdout:Sep 16 08:28:33 smithi193 podman[89015]: 2024-09-16 08:28:33.855869743 +0000 UTC m=+0.637706130 container died a26dd794dd843e576d215c07d4de3dc70e875fa79321d44755a6c9b11d93e60f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T08:28:34.422 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T08:28:34.462 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:28:34.462 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T08:28:34.463 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T08:28:34.463 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.7 2024-09-16T08:28:34.752 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:34 smithi193 systemd[1]: Stopping Ceph osd.7 for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:28:35.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:34 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:34.793+0000 7f1191927640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:28:35.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:34 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:34.793+0000 7f1191927640 -1 osd.7 104 *** Got signal Terminated *** 2024-09-16T08:28:35.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:34 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:34.793+0000 7f1191927640 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T08:28:35.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:34 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:34.877+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:35.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:34 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:34.877+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:35.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:34 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:34.877+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:35.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:34 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:34.877+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:36.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:35 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:35.904+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:36.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:35 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:35.904+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:36.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:35 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:35.904+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:36.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:35 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:35.904+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:37.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:36 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:36.897+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:37.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:36 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:36.897+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:37.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:36 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:36.897+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:37.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:36 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:36.897+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:37.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:36 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:36.897+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.193:6804 osd.4 since back 2024-09-16T08:28:10.687073+0000 front 2024-09-16T08:28:10.687119+0000 (oldest deadline 2024-09-16T08:28:35.987113+0000) 2024-09-16T08:28:38.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:37 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:37.935+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:38.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:37 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:37.935+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:38.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:37 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:37.935+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:38.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:37 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:37.935+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:38.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:37 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:37.935+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.193:6804 osd.4 since back 2024-09-16T08:28:10.687073+0000 front 2024-09-16T08:28:10.687119+0000 (oldest deadline 2024-09-16T08:28:35.987113+0000) 2024-09-16T08:28:39.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:38 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:38.949+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-16T08:27:36.481861+0000 front 2024-09-16T08:27:36.481749+0000 (oldest deadline 2024-09-16T08:28:01.181712+0000) 2024-09-16T08:28:39.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:38 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:38.949+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-09-16T08:27:47.583009+0000 front 2024-09-16T08:27:47.582998+0000 (oldest deadline 2024-09-16T08:28:09.883021+0000) 2024-09-16T08:28:39.252 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:38 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:38.949+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6822 osd.2 since back 2024-09-16T08:27:56.785097+0000 front 2024-09-16T08:27:56.785073+0000 (oldest deadline 2024-09-16T08:28:18.485083+0000) 2024-09-16T08:28:39.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:38 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:38.949+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.90:6830 osd.3 since back 2024-09-16T08:28:03.686340+0000 front 2024-09-16T08:28:03.686315+0000 (oldest deadline 2024-09-16T08:28:26.586345+0000) 2024-09-16T08:28:39.253 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:38 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7[42458]: 2024-09-16T08:28:38.949+0000 7f1189f33640 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.193:6804 osd.4 since back 2024-09-16T08:28:10.687073+0000 front 2024-09-16T08:28:10.687119+0000 (oldest deadline 2024-09-16T08:28:35.987113+0000) 2024-09-16T08:28:40.420 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:40 smithi193 podman[89117]: 2024-09-16 08:28:40.13601491 +0000 UTC m=+5.453278982 container died 9a7223ca10b194e5902e869b606a963a5cf086efa78146b820dbf067a713818f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T08:28:40.752 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:40 smithi193 podman[89117]: 2024-09-16 08:28:40.502686256 +0000 UTC m=+5.819950314 container cleanup 9a7223ca10b194e5902e869b606a963a5cf086efa78146b820dbf067a713818f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7, RELEASE=reef-ae7f373, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T08:28:40.752 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:40 smithi193 bash[89117]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7 2024-09-16T08:28:41.106 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:40 smithi193 podman[89133]: 2024-09-16 08:28:40.795589549 +0000 UTC m=+0.649251689 container remove 9a7223ca10b194e5902e869b606a963a5cf086efa78146b820dbf067a713818f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=reef-ae7f373) 2024-09-16T08:28:41.390 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:41 smithi193 podman[89224]: 2024-09-16 08:28:41.008873425 +0000 UTC m=+0.016022332 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:28:41.390 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:41 smithi193 podman[89224]: 2024-09-16 08:28:41.129660606 +0000 UTC m=+0.136809498 container create 5bcdd09374afcf30b08db1ba4a59983f20446b43a3910e4f85824699e60bdafb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-16T08:28:41.391 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:41 smithi193 podman[89224]: 2024-09-16 08:28:41.390507579 +0000 UTC m=+0.397656474 container init 5bcdd09374afcf30b08db1ba4a59983f20446b43a3910e4f85824699e60bdafb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=reef-ae7f373, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T08:28:41.752 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:41 smithi193 podman[89224]: 2024-09-16 08:28:41.394096322 +0000 UTC m=+0.401245223 container start 5bcdd09374afcf30b08db1ba4a59983f20446b43a3910e4f85824699e60bdafb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T08:28:41.752 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:41 smithi193 podman[89224]: 2024-09-16 08:28:41.47391727 +0000 UTC m=+0.481066170 container attach 5bcdd09374afcf30b08db1ba4a59983f20446b43a3910e4f85824699e60bdafb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T08:28:41.753 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:41 smithi193 podman[89224]: 2024-09-16 08:28:41.581020044 +0000 UTC m=+0.588168947 container died 5bcdd09374afcf30b08db1ba4a59983f20446b43a3910e4f85824699e60bdafb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T08:28:42.123 INFO:journalctl@ceph.osd.7.smithi193.stdout:Sep 16 08:28:42 smithi193 podman[89247]: 2024-09-16 08:28:42.10201635 +0000 UTC m=+0.515013696 container remove 5bcdd09374afcf30b08db1ba4a59983f20446b43a3910e4f85824699e60bdafb (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-16T08:28:42.129 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T08:28:42.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:28:42.166 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T08:28:42.166 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-16T08:28:42.166 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-8b378aa4-7403-11ef-bceb-c7b262605968@prometheus.a 2024-09-16T08:28:42.480 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 systemd[1]: Stopping Ceph prometheus.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:28:42.747 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T08:28:42.747 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-16T08:28:42.748 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-16T08:28:42.748 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T08:28:42.748 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-16T08:28:42.748 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-16T08:28:42.748 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T08:28:42.748 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-16T08:28:42.748 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-16T08:28:42.749 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T08:28:42.749 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-16T08:28:42.749 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a[80265]: ts=2024-09-16T08:28:42.480Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-16T08:28:42.749 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 podman[89326]: 2024-09-16 08:28:42.490525902 +0000 UTC m=+0.098521675 container died c0c16dcd857e5a0bdc076fc7371a8e45466048d11d0760e559706de1c771f75b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:28:43.002 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 podman[89326]: 2024-09-16 08:28:42.846437449 +0000 UTC m=+0.454433219 container cleanup c0c16dcd857e5a0bdc076fc7371a8e45466048d11d0760e559706de1c771f75b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T08:28:43.002 INFO:journalctl@ceph.prometheus.a.smithi193.stdout:Sep 16 08:28:42 smithi193 bash[89326]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T08:28:43.284 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8b378aa4-7403-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-16T08:28:43.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T08:28:43.318 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-16T08:28:43.318 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T08:28:43.461 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:28:45.243 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:44 smithi090 systemd[1]: Stopping Ceph alertmanager.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:28:45.244 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:45 smithi090 ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a[56403]: ts=2024-09-16T08:28:45.203Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T08:28:45.562 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:45 smithi090 podman[82393]: 2024-09-16 08:28:45.243941631 +0000 UTC m=+0.131955291 container died 3c66ea902fdc3b2c45526b3c322e0b93abcb5b1cd3546d9c228b59303eaf0137 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:28:45.922 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:45 smithi090 podman[82393]: 2024-09-16 08:28:45.645365135 +0000 UTC m=+0.533378849 container cleanup 3c66ea902fdc3b2c45526b3c322e0b93abcb5b1cd3546d9c228b59303eaf0137 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:28:45.922 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:45 smithi090 bash[82393]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T08:28:45.922 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:45 smithi090 podman[82405]: 2024-09-16 08:28:45.846662869 +0000 UTC m=+0.596999917 container remove 3c66ea902fdc3b2c45526b3c322e0b93abcb5b1cd3546d9c228b59303eaf0137 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T08:28:45.922 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:45 smithi090 podman[82405]: 2024-09-16 08:28:45.922386821 +0000 UTC m=+0.672723868 volume remove 76af2c5267aafb6b458ae6cfd1d82bc018a5f1da70c790f5ccbc914f9a029dda 2024-09-16T08:28:46.241 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:46 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T08:28:46.242 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:46 smithi090 systemd[1]: Stopped Ceph alertmanager.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:28:46.242 INFO:journalctl@ceph.alertmanager.a.smithi090.stdout:Sep 16 08:28:46 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.271s CPU time. 2024-09-16T08:28:46.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:28:46 smithi090 systemd[1]: Stopping Ceph node-exporter.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:28:46.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:28:46 smithi090 podman[82512]: 2024-09-16 08:28:46.486184062 +0000 UTC m=+0.104761342 container died 4a6ade533b17f416fdd3e3ff65c51810a414edb8ddfc91f7f748cecf266d7e15 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T08:28:47.061 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:28:46 smithi090 podman[82512]: 2024-09-16 08:28:46.818717644 +0000 UTC m=+0.437294918 container cleanup 4a6ade533b17f416fdd3e3ff65c51810a414edb8ddfc91f7f748cecf266d7e15 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T08:28:47.062 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:28:46 smithi090 bash[82512]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a 2024-09-16T08:28:47.062 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:28:46 smithi090 podman[82525]: 2024-09-16 08:28:46.970081868 +0000 UTC m=+0.486677696 container remove 4a6ade533b17f416fdd3e3ff65c51810a414edb8ddfc91f7f748cecf266d7e15 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T08:28:47.062 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:28:46 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T08:28:47.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:28:47 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-16T08:28:47.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:28:47 smithi090 systemd[1]: Stopped Ceph node-exporter.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:28:47.562 INFO:journalctl@ceph.node-exporter.a.smithi090.stdout:Sep 16 08:28:47 smithi090 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 2.449s CPU time. 2024-09-16T08:29:22.213 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T08:29:22.370 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:29:24.002 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:29:23 smithi193 systemd[1]: Stopping Ceph grafana.a for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:29:24.391 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:29:24 smithi193 podman[90726]: 2024-09-16 08:29:24.109884123 +0000 UTC m=+0.106095188 container died d2a25f536ef63bb6875efaca574e5e06cf75f33c16595f1385e78089f10436e5 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a, io.buildah.version=1.32.0, maintainer=Paul Cuzner , io.openshift.expose-services=, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., io.openshift.tags=base rhel8, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, release=239.1651231664, version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-09-16T08:29:24.752 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:29:24 smithi193 podman[90726]: 2024-09-16 08:29:24.557364266 +0000 UTC m=+0.553575368 container cleanup d2a25f536ef63bb6875efaca574e5e06cf75f33c16595f1385e78089f10436e5 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a, io.buildah.version=1.32.0, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, vendor=Red Hat, Inc., release=239.1651231664, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-16T08:29:24.752 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:29:24 smithi193 bash[90726]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a 2024-09-16T08:29:25.175 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:29:24 smithi193 podman[90738]: 2024-09-16 08:29:24.850231707 +0000 UTC m=+0.738073878 container remove d2a25f536ef63bb6875efaca574e5e06cf75f33c16595f1385e78089f10436e5 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, release=239.1651231664, maintainer=Paul Cuzner , version=8.5, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, vendor=Red Hat, Inc., description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-09-16T08:29:25.175 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:29:24 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T08:29:25.176 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:29:24 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-16T08:29:25.176 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:29:24 smithi193 systemd[1]: Stopped Ceph grafana.a for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:29:25.176 INFO:journalctl@ceph.grafana.a.smithi193.stdout:Sep 16 08:29:24 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@grafana.a.service: Consumed 10.733s CPU time. 2024-09-16T08:29:25.430 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:29:25 smithi193 systemd[1]: Stopping Ceph node-exporter.b for 8b378aa4-7403-11ef-bceb-c7b262605968... 2024-09-16T08:29:25.704 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:29:25 smithi193 podman[90845]: 2024-09-16 08:29:25.430362225 +0000 UTC m=+0.117822218 container died 46a72e78b0a242221f7e1e0acdb959b162853ca71015c25d0e391e3d13ac775c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T08:29:26.002 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:29:25 smithi193 podman[90845]: 2024-09-16 08:29:25.803039719 +0000 UTC m=+0.490499709 container cleanup 46a72e78b0a242221f7e1e0acdb959b162853ca71015c25d0e391e3d13ac775c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T08:29:26.002 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:29:25 smithi193 bash[90845]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b 2024-09-16T08:29:26.371 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:29:26 smithi193 podman[90857]: 2024-09-16 08:29:26.062367718 +0000 UTC m=+0.635462329 container remove 46a72e78b0a242221f7e1e0acdb959b162853ca71015c25d0e391e3d13ac775c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-8b378aa4-7403-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T08:29:26.372 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:29:26 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-16T08:29:26.372 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:29:26 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-16T08:29:26.372 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:29:26 smithi193 systemd[1]: Stopped Ceph node-exporter.b for 8b378aa4-7403-11ef-bceb-c7b262605968. 2024-09-16T08:29:26.372 INFO:journalctl@ceph.node-exporter.b.smithi193.stdout:Sep 16 08:29:26 smithi193 systemd[1]: ceph-8b378aa4-7403-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 2.420s CPU time. 2024-09-16T08:29:49.502 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:29:49.536 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:29:49.565 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T08:29:49.567 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906926/remote/smithi090/crash 2024-09-16T08:29:49.568 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/crash -- . 2024-09-16T08:29:49.650 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T08:29:49.650 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-09-16T08:29:49.652 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906926/remote/smithi193/crash 2024-09-16T08:29:49.653 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/crash -- . 2024-09-16T08:29:49.693 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T08:29:49.693 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-09-16T08:29:49.694 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T08:29:49.695 DEBUG:teuthology.orchestra.run.smithi090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v CEPHADM_AGENT_DOWN | head -n 1 2024-09-16T08:29:49.726 INFO:tasks.cephadm:Compressing logs... 2024-09-16T08:29:49.727 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T08:29:49.769 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T08:29:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T08:29:49.793 INFO:teuthology.orchestra.run.smithi090.stderr:: No such file or directory 2024-09-16T08:29:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.foo.smithi090.fbrjws.log 2024-09-16T08:29:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/tcmu-runner.log 2024-09-16T08:29:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.foo.smithi090.fbrjws.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T08:29:49.796 INFO:teuthology.orchestra.run.smithi090.stderr: 62.1% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.foo.smithi090.fbrjws.log.gz 2024-09-16T08:29:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T08:29:49.796 INFO:teuthology.orchestra.run.smithi090.stderr: 81.1% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/tcmu-runner.log.gz 2024-09-16T08:29:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.smpl.smithi090.ebelfk.log 2024-09-16T08:29:49.797 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.1.log: /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T08:29:49.797 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.smpl.smithi090.ebelfk.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T08:29:49.797 INFO:teuthology.orchestra.run.smithi090.stderr: 62.2% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.smpl.smithi090.ebelfk.log.gz 2024-09-16T08:29:49.797 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T08:29:49.797 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T08:29:49.798 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T08:29:49.799 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-09-16T08:29:49.799 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T08:29:49.799 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T08:29:49.799 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.smpl.smithi193.cgoeeh.log 2024-09-16T08:29:49.800 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T08:29:49.800 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mon.a.log: /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.2.log: 92.9% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T08:29:49.800 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T08:29:49.803 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T08:29:49.803 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.smpl.smithi193.cgoeeh.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T08:29:49.803 INFO:teuthology.orchestra.run.smithi193.stderr: 62.1% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.smpl.smithi193.cgoeeh.log.gz 2024-09-16T08:29:49.804 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.foo.smithi193.gjyvkw.log 2024-09-16T08:29:49.804 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T08:29:49.804 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.foo.smithi193.gjyvkw.log: 62.2% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-client.rgw.foo.smithi193.gjyvkw.log.gz 2024-09-16T08:29:49.804 INFO:teuthology.orchestra.run.smithi193.stderr: 87.5% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T08:29:49.804 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T08:29:49.804 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.log 2024-09-16T08:29:49.804 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.5.log: 91.7% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T08:29:49.805 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.log 2024-09-16T08:29:49.805 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mgr.y.log: 94.8% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T08:29:49.807 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T08:29:49.808 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.log: 93.4% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T08:29:49.809 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T08:29:49.812 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.log: 88.9% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T08:29:49.812 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T08:29:49.812 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T08:29:49.821 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T08:29:49.840 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 85.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T08:29:49.844 INFO:teuthology.orchestra.run.smithi090.stderr: 91.4% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T08:29:49.859 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 80.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T08:29:49.870 INFO:teuthology.orchestra.run.smithi193.stderr: 91.4% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T08:29:49.877 INFO:teuthology.orchestra.run.smithi193.stderr: 88.3% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T08:29:50.129 INFO:teuthology.orchestra.run.smithi193.stderr: 92.4% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T08:29:50.141 INFO:teuthology.orchestra.run.smithi090.stderr: 89.4% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T08:29:50.153 INFO:teuthology.orchestra.run.smithi090.stderr: 92.6% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T08:29:50.229 INFO:teuthology.orchestra.run.smithi090.stderr: 93.4% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T08:29:50.302 INFO:teuthology.orchestra.run.smithi193.stderr: 93.2% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T08:29:50.359 INFO:teuthology.orchestra.run.smithi193.stderr: 93.4% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T08:29:50.366 INFO:teuthology.orchestra.run.smithi193.stderr: 93.8% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T08:29:50.376 INFO:teuthology.orchestra.run.smithi090.stderr: 93.6% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T08:29:50.392 INFO:teuthology.orchestra.run.smithi193.stderr: 93.6% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T08:29:50.394 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-09-16T08:29:50.395 INFO:teuthology.orchestra.run.smithi193.stderr:real 0m0.611s 2024-09-16T08:29:50.395 INFO:teuthology.orchestra.run.smithi193.stderr:user 0m2.472s 2024-09-16T08:29:50.395 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m0.123s 2024-09-16T08:29:50.408 INFO:teuthology.orchestra.run.smithi090.stderr: 93.4% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T08:29:50.428 INFO:teuthology.orchestra.run.smithi090.stderr: 93.5% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T08:29:50.528 INFO:teuthology.orchestra.run.smithi090.stderr: 91.4% -- replaced with /var/log/ceph/8b378aa4-7403-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T08:29:50.530 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-09-16T08:29:50.530 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m0.748s 2024-09-16T08:29:50.530 INFO:teuthology.orchestra.run.smithi090.stderr:user 0m3.627s 2024-09-16T08:29:50.531 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m0.141s 2024-09-16T08:29:50.531 INFO:tasks.cephadm:Archiving logs... 2024-09-16T08:29:50.531 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906926/remote/smithi090/log 2024-09-16T08:29:50.532 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T08:29:51.098 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906926/remote/smithi193/log 2024-09-16T08:29:51.099 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T08:29:51.489 INFO:tasks.cephadm:Removing cluster... 2024-09-16T08:29:51.489 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 --force 2024-09-16T08:29:51.634 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:29:51.765 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 --force 2024-09-16T08:29:51.916 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 8b378aa4-7403-11ef-bceb-c7b262605968 2024-09-16T08:29:52.057 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T08:29:52.058 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T08:29:52.074 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T08:29:52.090 INFO:tasks.cephadm:Teardown complete 2024-09-16T08:29:52.091 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T08:29:52.101 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T08:29:52.101 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T08:29:52.116 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T08:29:52.128 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-09-16T08:29:52.133 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T08:29:52.133 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-09-16T08:29:52.133 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv01.front.sepia.ceph.com 3 6 377 10 +1276us[+1317us] +/- 51ms 2024-09-16T08:29:52.133 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -2638us[-2596us] +/- 53ms 2024-09-16T08:29:52.133 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +1411us[+1452us] +/- 62ms 2024-09-16T08:29:52.133 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:29:52.144 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-09-16T08:29:52.148 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T08:29:52.148 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-09-16T08:29:52.148 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 +1331us[+1456us] +/- 51ms 2024-09-16T08:29:52.148 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 -2581us[-2581us] +/- 53ms 2024-09-16T08:29:52.148 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +1469us[+1469us] +/- 62ms 2024-09-16T08:29:52.149 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:29:52.150 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T08:29:52.157 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T08:29:52.158 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T08:29:52.165 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T08:29:52.246 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T08:29:52.318 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T08:29:52.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:29:52.346 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T08:29:52.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:29:52.374 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T08:29:52.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T08:29:52.391 INFO:teuthology.task.internal:Duration was 1217.839413 seconds 2024-09-16T08:29:52.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T08:29:52.399 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T08:29:52.399 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T08:29:52.402 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T08:29:52.437 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T08:29:52.453 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T08:29:52.791 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T08:29:52.791 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-09-16T08:29:52.792 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T08:29:52.812 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-09-16T08:29:52.812 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T08:29:52.859 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T08:29:52.859 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T08:29:52.862 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T08:29:52.926 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T08:29:52.926 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T08:29:52.928 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T08:29:53.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T08:29:53.280 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T08:29:53.281 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T08:29:53.305 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T08:29:53.330 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T08:29:53.339 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T08:29:53.348 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T08:29:53.368 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-09-16T08:29:53.393 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-09-16T08:29:53.417 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T08:29:53.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:29:53.452 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T08:29:53.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:29:53.474 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T08:29:53.484 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T08:29:53.485 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906926/remote/smithi090 2024-09-16T08:29:53.485 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T08:29:53.534 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906926/remote/smithi193 2024-09-16T08:29:53.534 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T08:29:53.573 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T08:29:53.573 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T08:29:53.576 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T08:29:53.622 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T08:29:53.632 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T08:29:53.632 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T08:29:53.641 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T08:29:53.642 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T08:29:53.644 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T08:29:53.656 INFO:teuthology.orchestra.run.smithi090.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 08:29 /home/ubuntu/cephtest 2024-09-16T08:29:53.677 INFO:teuthology.orchestra.run.smithi193.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 08:29 /home/ubuntu/cephtest 2024-09-16T08:29:53.678 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T08:29:53.693 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T08:29:53.717 INFO:teuthology.run:Summary data: description: orch:cephadm/upgrade/{1-start-distro/1-start-centos_9.stream-reef 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} duration: 1217.8394131660461 failure_reason: 'Command failed on smithi090 with status 1: ''sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8b378aa4-7403-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c \''ceph versions | jq -e \''"\''"\''.overall | length == 1\''"\''"\''\''''' owner: scheduled_gabrioux@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=482547eb6dee47b48c8bbd4b6e9cf9f5 status: fail success: false 2024-09-16T08:29:53.717 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T08:29:53.803 INFO:teuthology.run:FAIL