2024-06-30T12:53:41.270 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-30T12:53:41.270 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592 --verbose --name teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi --description orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} -- /home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592/orig.config.yaml 2024-06-30T12:53:41.298 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-30T12:53:41.383 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592 branch: reef description: orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} email: ceph-qa@ceph.com first_in_suite: false job_id: '7749592' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi no_nested_subset: false os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: reef ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 workunit: branch: reef sha1: 16b16b98599950a27899283412b0a32cd03699ac owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b seed: 9459 sha1: f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 sleep_before_teardown: 0 subset: 38/64 suite: orch suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_16b16b98599950a27899283412b0a32cd03699ac/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 16b16b98599950a27899283412b0a32cd03699ac targets: smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - cephadm: allow_ptrace: false avoid_pacific_features: true cephadm_branch: v16.2.0 cephadm_git_url: https://github.com/ceph/ceph image: quay.io/ceph/ceph:v16.2.0 - cephadm.shell: env: - sha1 mon.a: - radosgw-admin realm create --rgw-realm=r --default - radosgw-admin zonegroup create --rgw-zonegroup=default --master --default - radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default - radosgw-admin period update --rgw-realm=r --commit - ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000 - ceph orch apply rgw smpl - ceph osd pool create foo - rbd pool init foo - ceph orch apply iscsi foo u p - sleep 120 - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 mon.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph orch upgrade status - ceph health detail - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 - ceph orch ls | grep '^osd ' - cephadm.shell: mon.a: - ceph orch upgrade ls - ceph orch upgrade ls --image quay.io/ceph/ceph --show-all-versions | grep 16.2.0 - ceph orch upgrade ls --image quay.io/ceph/ceph --tags | grep v16.2.2 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-10_22:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698093 2024-06-30T12:53:41.383 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_16b16b98599950a27899283412b0a32cd03699ac/qa; will attempt to use it 2024-06-30T12:53:41.384 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_16b16b98599950a27899283412b0a32cd03699ac/qa/tasks 2024-06-30T12:53:41.385 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-30T12:53:41.386 INFO:teuthology.task.internal:Checking packages... 2024-06-30T12:53:41.406 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f1abf795f4fc15bf2f30ac90c811a53ab21d5d65' 2024-06-30T12:53:41.406 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-30T12:53:41.407 INFO:teuthology.packaging:ref: None 2024-06-30T12:53:41.407 INFO:teuthology.packaging:tag: None 2024-06-30T12:53:41.407 INFO:teuthology.packaging:branch: reef 2024-06-30T12:53:41.407 INFO:teuthology.packaging:sha1: f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T12:53:41.407 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=reef 2024-06-30T12:53:41.720 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1561-gbec9e3ef-1focal 2024-06-30T12:53:41.722 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-30T12:53:41.728 INFO:teuthology.task.internal:no buildpackages task found 2024-06-30T12:53:41.728 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-30T12:53:41.751 INFO:teuthology.task.internal:Saving configuration 2024-06-30T12:53:41.762 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-30T12:53:41.768 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-30T12:53:41.794 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-30 12:47:35.835824', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-06-30T12:53:41.817 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-30 12:47:35.834376', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-06-30T12:53:41.818 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-30T12:53:41.823 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2024-06-30T12:53:41.823 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-06-30T12:53:41.823 INFO:teuthology.run_tasks:Running task console_log... 2024-06-30T12:53:41.894 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb5359cd120>, signals=[15]) 2024-06-30T12:53:41.894 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-30T12:53:41.900 INFO:teuthology.task.internal:Opening connections... 2024-06-30T12:53:41.900 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-06-30T12:53:41.901 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T12:53:42.000 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-06-30T12:53:42.001 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T12:53:42.086 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-30T12:53:42.092 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-06-30T12:53:42.099 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-06-30T12:53:42.099 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-06-30T12:53:42.150 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Ubuntu" 2024-06-30T12:53:42.150 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-06-30T12:53:42.150 INFO:teuthology.orchestra.run.smithi144.stdout:ID=ubuntu 2024-06-30T12:53:42.150 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE=debian 2024-06-30T12:53:42.151 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-06-30T12:53:42.151 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="20.04" 2024-06-30T12:53:42.151 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.ubuntu.com/" 2024-06-30T12:53:42.151 INFO:teuthology.orchestra.run.smithi144.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-06-30T12:53:42.151 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-06-30T12:53:42.151 INFO:teuthology.orchestra.run.smithi144.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-06-30T12:53:42.151 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_CODENAME=focal 2024-06-30T12:53:42.151 INFO:teuthology.orchestra.run.smithi144.stdout:UBUNTU_CODENAME=focal 2024-06-30T12:53:42.152 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-06-30T12:53:42.169 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-06-30T12:53:42.176 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-06-30T12:53:42.176 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-06-30T12:53:42.228 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Ubuntu" 2024-06-30T12:53:42.228 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:ID=ubuntu 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE=debian 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="20.04" 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.ubuntu.com/" 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_CODENAME=focal 2024-06-30T12:53:42.229 INFO:teuthology.orchestra.run.smithi167.stdout:UBUNTU_CODENAME=focal 2024-06-30T12:53:42.229 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-06-30T12:53:42.253 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-30T12:53:42.261 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-30T12:53:42.274 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-30T12:53:42.274 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-06-30T12:53:42.277 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-06-30T12:53:42.282 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-30T12:53:42.292 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-30T12:53:42.292 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-06-30T12:53:42.294 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-06-30T12:53:42.315 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-30T12:53:42.366 INFO:teuthology.run_tasks:Running task kernel... 2024-06-30T12:53:42.379 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-30T12:53:42.379 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-30T12:53:42.379 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-30T12:53:42.380 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-30T12:53:42.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-30T12:53:42.385 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-06-30T12:53:42.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-30T12:53:42.386 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-06-30T12:53:42.392 INFO:teuthology.orchestra.run.smithi144.stdout:5.4.0-124-generic 2024-06-30T12:53:42.392 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.4.0-124-generic 2024-06-30T12:53:42.392 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get clean 2024-06-30T12:53:42.393 INFO:teuthology.orchestra.run.smithi167.stdout:5.4.0-124-generic 2024-06-30T12:53:42.394 INFO:teuthology.task.kernel:Running kernel on smithi167: 5.4.0-124-generic 2024-06-30T12:53:42.394 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get clean 2024-06-30T12:53:42.539 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-06-30T12:53:42.570 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2024-06-30T12:53:42.696 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-06-30T12:53:42.730 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-06-30T12:53:42.804 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-06-30T12:53:42.827 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-06-30T12:53:42.886 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-06-30T12:53:42.909 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-06-30T12:53:42.994 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,020 kB] 2024-06-30T12:53:43.034 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [769 kB] 2024-06-30T12:53:43.128 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,020 kB] 2024-06-30T12:53:43.180 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [452 kB] 2024-06-30T12:53:43.185 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-06-30T12:53:43.185 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,924 kB] 2024-06-30T12:53:43.191 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [769 kB] 2024-06-30T12:53:43.204 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [452 kB] 2024-06-30T12:53:43.212 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2024-06-30T12:53:43.212 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,924 kB] 2024-06-30T12:53:43.239 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-06-30T12:53:43.242 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-06-30T12:53:43.243 INFO:teuthology.orchestra.run.smithi167.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [409 kB] 2024-06-30T12:53:43.247 INFO:teuthology.orchestra.run.smithi167.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-06-30T12:53:43.247 INFO:teuthology.orchestra.run.smithi167.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [980 kB] 2024-06-30T12:53:43.260 INFO:teuthology.orchestra.run.smithi167.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-06-30T12:53:43.262 INFO:teuthology.orchestra.run.smithi167.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [665 kB] 2024-06-30T12:53:43.269 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.1 kB] 2024-06-30T12:53:43.269 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [409 kB] 2024-06-30T12:53:43.277 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [552 B] 2024-06-30T12:53:43.277 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [665 kB] 2024-06-30T12:53:43.284 INFO:teuthology.orchestra.run.smithi167.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [207 kB] 2024-06-30T12:53:43.286 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [980 kB] 2024-06-30T12:53:43.286 INFO:teuthology.orchestra.run.smithi167.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-06-30T12:53:43.295 INFO:teuthology.orchestra.run.smithi167.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-06-30T12:53:43.295 INFO:teuthology.orchestra.run.smithi167.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-06-30T12:53:43.298 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [207 kB] 2024-06-30T12:53:43.301 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-06-30T12:53:43.301 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-06-30T12:53:43.301 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-06-30T12:53:43.316 INFO:teuthology.orchestra.run.smithi167.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-06-30T12:53:43.316 INFO:teuthology.orchestra.run.smithi167.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-06-30T12:53:43.323 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-06-30T12:53:43.323 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2024-06-30T12:53:43.380 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [993 kB] 2024-06-30T12:53:43.401 INFO:teuthology.orchestra.run.smithi167.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,392 kB] 2024-06-30T12:53:43.619 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,392 kB] 2024-06-30T12:53:43.779 INFO:teuthology.orchestra.run.smithi167.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [993 kB] 2024-06-30T12:53:43.786 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [531 kB] 2024-06-30T12:53:43.799 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-06-30T12:53:43.800 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,040 kB] 2024-06-30T12:53:43.807 INFO:teuthology.orchestra.run.smithi167.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [531 kB] 2024-06-30T12:53:43.820 INFO:teuthology.orchestra.run.smithi167.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.2 kB] 2024-06-30T12:53:43.821 INFO:teuthology.orchestra.run.smithi167.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-06-30T12:53:43.822 INFO:teuthology.orchestra.run.smithi167.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,040 kB] 2024-06-30T12:53:43.885 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.4 kB] 2024-06-30T12:53:43.885 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [425 kB] 2024-06-30T12:53:43.891 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-06-30T12:53:43.891 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [791 kB] 2024-06-30T12:53:43.908 INFO:teuthology.orchestra.run.smithi167.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [425 kB] 2024-06-30T12:53:43.914 INFO:teuthology.orchestra.run.smithi167.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [552 B] 2024-06-30T12:53:43.914 INFO:teuthology.orchestra.run.smithi167.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [791 kB] 2024-06-30T12:53:43.928 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,199 kB] 2024-06-30T12:53:43.942 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [289 kB] 2024-06-30T12:53:43.950 INFO:teuthology.orchestra.run.smithi167.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,199 kB] 2024-06-30T12:53:43.964 INFO:teuthology.orchestra.run.smithi167.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [289 kB] 2024-06-30T12:53:43.965 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-06-30T12:53:43.966 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-06-30T12:53:43.966 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-06-30T12:53:43.987 INFO:teuthology.orchestra.run.smithi167.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.7 kB] 2024-06-30T12:53:43.987 INFO:teuthology.orchestra.run.smithi167.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-06-30T12:53:43.988 INFO:teuthology.orchestra.run.smithi167.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-06-30T12:53:43.988 INFO:teuthology.orchestra.run.smithi167.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-06-30T12:53:44.024 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-06-30T12:53:44.024 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-06-30T12:53:44.025 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-06-30T12:53:44.025 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-06-30T12:53:44.026 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-06-30T12:53:44.031 INFO:teuthology.orchestra.run.smithi167.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2024-06-30T12:53:44.031 INFO:teuthology.orchestra.run.smithi167.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-06-30T12:53:44.032 INFO:teuthology.orchestra.run.smithi167.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-06-30T12:53:44.046 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-06-30T12:53:44.112 INFO:teuthology.orchestra.run.smithi167.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-06-30T12:53:44.112 INFO:teuthology.orchestra.run.smithi167.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-06-30T12:53:44.112 INFO:teuthology.orchestra.run.smithi167.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-06-30T12:53:44.113 INFO:teuthology.orchestra.run.smithi167.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-06-30T12:53:44.127 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-06-30T12:53:44.128 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-06-30T12:53:47.278 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 20.8 MB in 3s (7,452 kB/s) 2024-06-30T12:53:47.532 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 20.8 MB in 3s (7,218 kB/s) 2024-06-30T12:53:48.731 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-06-30T12:53:48.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-06-30T12:53:48.828 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-06-30T12:53:49.017 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2024-06-30T12:53:49.018 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2024-06-30T12:53:49.116 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-06-30T12:53:49.143 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-06-30T12:53:49.207 INFO:teuthology.orchestra.run.smithi167.stdout:The following additional packages will be installed: 2024-06-30T12:53:49.207 INFO:teuthology.orchestra.run.smithi167.stdout: linux-generic linux-headers-5.4.0-187 linux-headers-5.4.0-187-generic 2024-06-30T12:53:49.207 INFO:teuthology.orchestra.run.smithi167.stdout: linux-headers-generic linux-image-5.4.0-187-generic 2024-06-30T12:53:49.207 INFO:teuthology.orchestra.run.smithi167.stdout: linux-modules-5.4.0-187-generic linux-modules-extra-5.4.0-187-generic 2024-06-30T12:53:49.208 INFO:teuthology.orchestra.run.smithi167.stdout:Suggested packages: 2024-06-30T12:53:49.208 INFO:teuthology.orchestra.run.smithi167.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-06-30T12:53:49.209 INFO:teuthology.orchestra.run.smithi167.stdout:Recommended packages: 2024-06-30T12:53:49.209 INFO:teuthology.orchestra.run.smithi167.stdout: thermald 2024-06-30T12:53:49.214 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-06-30T12:53:49.261 INFO:teuthology.orchestra.run.smithi167.stdout:The following NEW packages will be installed: 2024-06-30T12:53:49.261 INFO:teuthology.orchestra.run.smithi167.stdout: linux-headers-5.4.0-187 linux-headers-5.4.0-187-generic 2024-06-30T12:53:49.261 INFO:teuthology.orchestra.run.smithi167.stdout: linux-image-5.4.0-187-generic linux-modules-5.4.0-187-generic 2024-06-30T12:53:49.262 INFO:teuthology.orchestra.run.smithi167.stdout: linux-modules-extra-5.4.0-187-generic 2024-06-30T12:53:49.263 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages will be upgraded: 2024-06-30T12:53:49.263 INFO:teuthology.orchestra.run.smithi167.stdout: linux-generic linux-headers-generic linux-image-generic 2024-06-30T12:53:49.427 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-06-30T12:53:49.427 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-06-30T12:53:49.445 INFO:teuthology.orchestra.run.smithi167.stdout:3 upgraded, 5 newly installed, 0 to remove and 306 not upgraded. 2024-06-30T12:53:49.445 INFO:teuthology.orchestra.run.smithi167.stdout:Need to get 77.1 MB of archives. 2024-06-30T12:53:49.446 INFO:teuthology.orchestra.run.smithi167.stdout:After this operation, 380 MB of additional disk space will be used. 2024-06-30T12:53:49.446 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-187-generic amd64 5.4.0-187.207 [15.0 MB] 2024-06-30T12:53:49.624 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2024-06-30T12:53:49.624 INFO:teuthology.orchestra.run.smithi144.stdout: linux-generic linux-headers-5.4.0-187 linux-headers-5.4.0-187-generic 2024-06-30T12:53:49.624 INFO:teuthology.orchestra.run.smithi144.stdout: linux-headers-generic linux-image-5.4.0-187-generic 2024-06-30T12:53:49.625 INFO:teuthology.orchestra.run.smithi144.stdout: linux-modules-5.4.0-187-generic linux-modules-extra-5.4.0-187-generic 2024-06-30T12:53:49.626 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2024-06-30T12:53:49.626 INFO:teuthology.orchestra.run.smithi144.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-06-30T12:53:49.626 INFO:teuthology.orchestra.run.smithi144.stdout:Recommended packages: 2024-06-30T12:53:49.626 INFO:teuthology.orchestra.run.smithi144.stdout: thermald 2024-06-30T12:53:49.681 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2024-06-30T12:53:49.681 INFO:teuthology.orchestra.run.smithi144.stdout: linux-headers-5.4.0-187 linux-headers-5.4.0-187-generic 2024-06-30T12:53:49.681 INFO:teuthology.orchestra.run.smithi144.stdout: linux-image-5.4.0-187-generic linux-modules-5.4.0-187-generic 2024-06-30T12:53:49.681 INFO:teuthology.orchestra.run.smithi144.stdout: linux-modules-extra-5.4.0-187-generic 2024-06-30T12:53:49.682 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages will be upgraded: 2024-06-30T12:53:49.683 INFO:teuthology.orchestra.run.smithi144.stdout: linux-generic linux-headers-generic linux-image-generic 2024-06-30T12:53:49.747 INFO:teuthology.orchestra.run.smithi144.stdout:3 upgraded, 5 newly installed, 0 to remove and 306 not upgraded. 2024-06-30T12:53:49.748 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 77.1 MB of archives. 2024-06-30T12:53:49.748 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 380 MB of additional disk space will be used. 2024-06-30T12:53:49.748 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-187-generic amd64 5.4.0-187.207 [15.0 MB] 2024-06-30T12:53:50.183 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-187-generic amd64 5.4.0-187.207 [10.5 MB] 2024-06-30T12:53:50.322 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-187-generic amd64 5.4.0-187.207 [39.2 MB] 2024-06-30T12:53:50.593 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-187-generic amd64 5.4.0-187.207 [10.5 MB] 2024-06-30T12:53:50.916 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-187-generic amd64 5.4.0-187.207 [39.2 MB] 2024-06-30T12:53:50.962 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.187.185 [1,900 B] 2024-06-30T12:53:50.962 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.187.185 [2,596 B] 2024-06-30T12:53:50.963 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-187 all 5.4.0-187.207 [11.0 MB] 2024-06-30T12:53:51.089 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-187-generic amd64 5.4.0-187.207 [1,383 kB] 2024-06-30T12:53:51.105 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.187.185 [2,468 B] 2024-06-30T12:53:51.918 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 77.1 MB in 1s (51.7 MB/s) 2024-06-30T12:53:52.117 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-modules-5.4.0-187-generic. 2024-06-30T12:53:52.142 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.187.185 [1,900 B] 2024-06-30T12:53:52.142 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.187.185 [2,596 B] 2024-06-30T12:53:52.142 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-187 all 5.4.0-187.207 [11.0 MB] 2024-06-30T12:53:52.573 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-187-generic amd64 5.4.0-187.207 [1,383 kB] 2024-06-30T12:53:52.624 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.187.185 [2,468 B] 2024-06-30T12:53:53.313 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 77.1 MB in 3s (23.0 MB/s) 2024-06-30T12:53:53.497 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-modules-5.4.0-187-generic. 2024-06-30T12:53:53.857 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-06-30T12:53:53.861 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../0-linux-modules-5.4.0-187-generic_5.4.0-187.207_amd64.deb ... 2024-06-30T12:53:53.989 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-modules-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:53:55.196 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-06-30T12:53:55.201 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../0-linux-modules-5.4.0-187-generic_5.4.0-187.207_amd64.deb ... 2024-06-30T12:53:55.286 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-modules-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:53:56.306 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-image-5.4.0-187-generic. 2024-06-30T12:53:56.319 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../1-linux-image-5.4.0-187-generic_5.4.0-187.207_amd64.deb ... 2024-06-30T12:53:56.429 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-image-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:53:57.195 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-image-5.4.0-187-generic. 2024-06-30T12:53:57.215 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../1-linux-image-5.4.0-187-generic_5.4.0-187.207_amd64.deb ... 2024-06-30T12:53:57.320 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-image-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:53:57.526 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-187-generic. 2024-06-30T12:53:57.539 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-187-generic_5.4.0-187.207_amd64.deb ... 2024-06-30T12:53:57.576 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-modules-extra-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:53:58.349 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-187-generic. 2024-06-30T12:53:58.363 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-187-generic_5.4.0-187.207_amd64.deb ... 2024-06-30T12:53:58.399 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-modules-extra-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:02.499 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../3-linux-generic_5.4.0.187.185_amd64.deb ... 2024-06-30T12:54:02.622 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-generic (5.4.0.187.185) over (5.4.0.124.125) ... 2024-06-30T12:54:02.933 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.187.185_amd64.deb ... 2024-06-30T12:54:03.050 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-image-generic (5.4.0.187.185) over (5.4.0.124.125) ... 2024-06-30T12:54:03.427 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-headers-5.4.0-187. 2024-06-30T12:54:03.442 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../5-linux-headers-5.4.0-187_5.4.0-187.207_all.deb ... 2024-06-30T12:54:03.486 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-5.4.0-187 (5.4.0-187.207) ... 2024-06-30T12:54:03.522 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../3-linux-generic_5.4.0.187.185_amd64.deb ... 2024-06-30T12:54:03.649 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-generic (5.4.0.187.185) over (5.4.0.124.125) ... 2024-06-30T12:54:03.967 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.187.185_amd64.deb ... 2024-06-30T12:54:04.094 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-image-generic (5.4.0.187.185) over (5.4.0.124.125) ... 2024-06-30T12:54:04.439 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-headers-5.4.0-187. 2024-06-30T12:54:04.453 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../5-linux-headers-5.4.0-187_5.4.0-187.207_all.deb ... 2024-06-30T12:54:04.497 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-5.4.0-187 (5.4.0-187.207) ... 2024-06-30T12:54:09.115 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package linux-headers-5.4.0-187-generic. 2024-06-30T12:54:09.137 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../6-linux-headers-5.4.0-187-generic_5.4.0-187.207_amd64.deb ... 2024-06-30T12:54:09.181 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:09.721 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-headers-5.4.0-187-generic. 2024-06-30T12:54:09.744 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../6-linux-headers-5.4.0-187-generic_5.4.0-187.207_amd64.deb ... 2024-06-30T12:54:09.787 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:11.923 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.187.185_amd64.deb ... 2024-06-30T12:54:12.051 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking linux-headers-generic (5.4.0.187.185) over (5.4.0.124.125) ... 2024-06-30T12:54:12.411 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-5.4.0-187 (5.4.0-187.207) ... 2024-06-30T12:54:12.444 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.187.185_amd64.deb ... 2024-06-30T12:54:12.555 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-modules-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:12.571 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-generic (5.4.0.187.185) over (5.4.0.124.125) ... 2024-06-30T12:54:12.906 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-5.4.0-187 (5.4.0-187.207) ... 2024-06-30T12:54:13.035 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-modules-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:13.766 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:13.918 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-image-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:14.270 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:14.396 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-image-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:15.085 INFO:teuthology.orchestra.run.smithi144.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-187-generic 2024-06-30T12:54:15.085 INFO:teuthology.orchestra.run.smithi144.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-187-generic 2024-06-30T12:54:15.236 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-modules-extra-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:15.564 INFO:teuthology.orchestra.run.smithi167.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-187-generic 2024-06-30T12:54:15.564 INFO:teuthology.orchestra.run.smithi167.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-187-generic 2024-06-30T12:54:15.715 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-modules-extra-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:16.821 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-headers-generic (5.4.0.187.185) ... 2024-06-30T12:54:16.933 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-generic (5.4.0.187.185) ... 2024-06-30T12:54:16.955 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-image-generic (5.4.0.187.185) ... 2024-06-30T12:54:17.051 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-image-generic (5.4.0.187.185) ... 2024-06-30T12:54:17.081 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up linux-generic (5.4.0.187.185) ... 2024-06-30T12:54:17.177 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-generic (5.4.0.187.185) ... 2024-06-30T12:54:17.225 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for linux-image-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:17.290 INFO:teuthology.orchestra.run.smithi144.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-06-30T12:54:17.291 INFO:teuthology.orchestra.run.smithi144.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-187-generic 2024-06-30T12:54:17.303 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for linux-image-5.4.0-187-generic (5.4.0-187.207) ... 2024-06-30T12:54:17.375 INFO:teuthology.orchestra.run.smithi167.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-06-30T12:54:17.375 INFO:teuthology.orchestra.run.smithi167.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-187-generic 2024-06-30T12:54:39.095 INFO:teuthology.orchestra.run.smithi144.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-06-30T12:54:39.096 INFO:teuthology.orchestra.run.smithi144.stdout:Sourcing file `/etc/default/grub' 2024-06-30T12:54:39.117 INFO:teuthology.orchestra.run.smithi144.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-06-30T12:54:39.124 INFO:teuthology.orchestra.run.smithi144.stdout:Generating grub configuration file ... 2024-06-30T12:54:39.200 INFO:teuthology.orchestra.run.smithi167.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-06-30T12:54:39.200 INFO:teuthology.orchestra.run.smithi167.stdout:Sourcing file `/etc/default/grub' 2024-06-30T12:54:39.212 INFO:teuthology.orchestra.run.smithi167.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-06-30T12:54:39.241 INFO:teuthology.orchestra.run.smithi167.stdout:Generating grub configuration file ... 2024-06-30T12:54:39.481 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.4.0-187-generic 2024-06-30T12:54:39.505 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.4.0-187-generic 2024-06-30T12:54:39.605 INFO:teuthology.orchestra.run.smithi167.stdout:Found linux image: /boot/vmlinuz-5.4.0-187-generic 2024-06-30T12:54:39.630 INFO:teuthology.orchestra.run.smithi167.stdout:Found initrd image: /boot/initrd.img-5.4.0-187-generic 2024-06-30T12:54:39.927 INFO:teuthology.orchestra.run.smithi144.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-06-30T12:54:39.939 INFO:teuthology.orchestra.run.smithi144.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-06-30T12:54:40.045 INFO:teuthology.orchestra.run.smithi167.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-06-30T12:54:40.057 INFO:teuthology.orchestra.run.smithi167.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-06-30T12:54:41.742 INFO:teuthology.orchestra.run.smithi144.stdout:done 2024-06-30T12:54:41.815 INFO:teuthology.orchestra.run.smithi167.stdout:done 2024-06-30T12:54:42.181 DEBUG:teuthology.orchestra.run.smithi144:> dpkg -s linux-image-generic 2024-06-30T12:54:42.225 INFO:teuthology.orchestra.run.smithi144.stdout:Package: linux-image-generic 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Status: install ok installed 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Priority: optional 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Section: kernel 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Installed-Size: 20 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Maintainer: Ubuntu Kernel Team 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: amd64 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Source: linux-meta 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Version: 5.4.0.187.185 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Depends: linux-image-5.4.0-187-generic, linux-modules-extra-5.4.0-187-generic, linux-firmware, intel-microcode, amd64-microcode 2024-06-30T12:54:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Recommends: thermald 2024-06-30T12:54:42.227 INFO:teuthology.orchestra.run.smithi144.stdout:Description: Generic Linux kernel image 2024-06-30T12:54:42.227 INFO:teuthology.orchestra.run.smithi144.stdout: This package will always depend on the latest generic kernel image 2024-06-30T12:54:42.227 INFO:teuthology.orchestra.run.smithi144.stdout: available. 2024-06-30T12:54:42.227 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-187-generic 2024-06-30T12:54:42.227 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-30T12:54:42.227 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-30T12:54:42.227 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-06-30T12:54:42.227 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-06-30T12:54:42.227 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get clean 2024-06-30T12:54:42.231 DEBUG:teuthology.orchestra.run.smithi167:> dpkg -s linux-image-generic 2024-06-30T12:54:42.274 INFO:teuthology.orchestra.run.smithi167.stdout:Package: linux-image-generic 2024-06-30T12:54:42.274 INFO:teuthology.orchestra.run.smithi167.stdout:Status: install ok installed 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Priority: optional 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Section: kernel 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Installed-Size: 20 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Maintainer: Ubuntu Kernel Team 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: amd64 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Source: linux-meta 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Version: 5.4.0.187.185 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Depends: linux-image-5.4.0-187-generic, linux-modules-extra-5.4.0-187-generic, linux-firmware, intel-microcode, amd64-microcode 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Recommends: thermald 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout:Description: Generic Linux kernel image 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout: This package will always depend on the latest generic kernel image 2024-06-30T12:54:42.275 INFO:teuthology.orchestra.run.smithi167.stdout: available. 2024-06-30T12:54:42.276 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-187-generic 2024-06-30T12:54:42.276 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-30T12:54:42.276 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-30T12:54:42.276 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-06-30T12:54:42.276 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=None, version=distro) 2024-06-30T12:54:42.276 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get clean 2024-06-30T12:54:42.313 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2024-06-30T12:54:42.359 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2024-06-30T12:54:42.452 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-06-30T12:54:42.470 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-06-30T12:54:42.489 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-06-30T12:54:42.505 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-06-30T12:54:42.505 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-06-30T12:54:42.517 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-06-30T12:54:42.538 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-06-30T12:54:42.563 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-06-30T12:54:44.315 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-06-30T12:54:44.326 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-06-30T12:54:44.342 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-06-30T12:54:44.353 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-06-30T12:54:44.411 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2024-06-30T12:54:44.428 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-06-30T12:54:44.625 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2024-06-30T12:54:44.625 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2024-06-30T12:54:44.638 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2024-06-30T12:54:44.639 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2024-06-30T12:54:44.862 INFO:teuthology.orchestra.run.smithi144.stdout:linux-image-generic is already the newest version (5.4.0.187.185). 2024-06-30T12:54:44.862 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 0 newly installed, 0 to remove and 306 not upgraded. 2024-06-30T12:54:44.864 DEBUG:teuthology.orchestra.run.smithi144:> dpkg -s linux-image-generic 2024-06-30T12:54:44.864 INFO:teuthology.orchestra.run.smithi167.stdout:linux-image-generic is already the newest version (5.4.0.187.185). 2024-06-30T12:54:44.865 INFO:teuthology.orchestra.run.smithi167.stdout:0 upgraded, 0 newly installed, 0 to remove and 306 not upgraded. 2024-06-30T12:54:44.870 DEBUG:teuthology.orchestra.run.smithi167:> dpkg -s linux-image-generic 2024-06-30T12:54:44.887 INFO:teuthology.orchestra.run.smithi144.stdout:Package: linux-image-generic 2024-06-30T12:54:44.887 INFO:teuthology.orchestra.run.smithi144.stdout:Status: install ok installed 2024-06-30T12:54:44.887 INFO:teuthology.orchestra.run.smithi144.stdout:Priority: optional 2024-06-30T12:54:44.887 INFO:teuthology.orchestra.run.smithi144.stdout:Section: kernel 2024-06-30T12:54:44.887 INFO:teuthology.orchestra.run.smithi144.stdout:Installed-Size: 20 2024-06-30T12:54:44.887 INFO:teuthology.orchestra.run.smithi144.stdout:Maintainer: Ubuntu Kernel Team 2024-06-30T12:54:44.888 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: amd64 2024-06-30T12:54:44.888 INFO:teuthology.orchestra.run.smithi144.stdout:Source: linux-meta 2024-06-30T12:54:44.888 INFO:teuthology.orchestra.run.smithi144.stdout:Version: 5.4.0.187.185 2024-06-30T12:54:44.888 INFO:teuthology.orchestra.run.smithi144.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-06-30T12:54:44.888 INFO:teuthology.orchestra.run.smithi144.stdout:Depends: linux-image-5.4.0-187-generic, linux-modules-extra-5.4.0-187-generic, linux-firmware, intel-microcode, amd64-microcode 2024-06-30T12:54:44.888 INFO:teuthology.orchestra.run.smithi144.stdout:Recommends: thermald 2024-06-30T12:54:44.888 INFO:teuthology.orchestra.run.smithi144.stdout:Description: Generic Linux kernel image 2024-06-30T12:54:44.888 INFO:teuthology.orchestra.run.smithi144.stdout: This package will always depend on the latest generic kernel image 2024-06-30T12:54:44.888 INFO:teuthology.orchestra.run.smithi144.stdout: available. 2024-06-30T12:54:44.888 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-06-30T12:54:44.893 INFO:teuthology.orchestra.run.smithi167.stdout:Package: linux-image-generic 2024-06-30T12:54:44.893 INFO:teuthology.orchestra.run.smithi167.stdout:Status: install ok installed 2024-06-30T12:54:44.893 INFO:teuthology.orchestra.run.smithi167.stdout:Priority: optional 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Section: kernel 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Installed-Size: 20 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Maintainer: Ubuntu Kernel Team 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: amd64 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Source: linux-meta 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Version: 5.4.0.187.185 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Depends: linux-image-5.4.0-187-generic, linux-modules-extra-5.4.0-187-generic, linux-firmware, intel-microcode, amd64-microcode 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Recommends: thermald 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout:Description: Generic Linux kernel image 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout: This package will always depend on the latest generic kernel image 2024-06-30T12:54:44.894 INFO:teuthology.orchestra.run.smithi167.stdout: available. 2024-06-30T12:54:44.895 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-06-30T12:54:44.899 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.UYXjHwxkEU 2024-06-30T12:54:44.899 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub/grub.cfg /tmp/tmp.UYXjHwxkEU 2024-06-30T12:54:44.936 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.a4EA48tDmn 2024-06-30T12:54:44.936 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub/grub.cfg /tmp/tmp.a4EA48tDmn 2024-06-30T12:54:44.956 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.UYXjHwxkEU 2024-06-30T12:54:44.957 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.a4EA48tDmn 2024-06-30T12:54:44.989 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.UYXjHwxkEU is 9KB 2024-06-30T12:54:44.999 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.a4EA48tDmn is 9KB 2024-06-30T12:54:45.004 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.UYXjHwxkEU 2024-06-30T12:54:45.011 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-06-30T12:54:45.015 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.a4EA48tDmn 2024-06-30T12:54:45.021 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-06-30T12:54:45.067 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-30T12:54:45.068 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-06-30T12:54:45.068 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-06-30T12:54:45.078 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T12:54:45.078 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-06-30T12:54:45.078 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-06-30T12:54:45.105 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-187-generic 2024-06-30T12:54:45.105 DEBUG:teuthology.orchestra.run.smithi144:> sudo update-grub 2024-06-30T12:54:45.126 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-187-generic 2024-06-30T12:54:45.126 DEBUG:teuthology.orchestra.run.smithi167:> sudo update-grub 2024-06-30T12:54:46.427 INFO:teuthology.orchestra.run.smithi167.stderr:Sourcing file `/etc/default/grub' 2024-06-30T12:54:46.431 INFO:teuthology.orchestra.run.smithi167.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-06-30T12:54:46.440 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2024-06-30T12:54:46.569 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub' 2024-06-30T12:54:46.573 INFO:teuthology.orchestra.run.smithi144.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-06-30T12:54:46.583 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-06-30T12:54:46.745 INFO:teuthology.orchestra.run.smithi167.stderr:Found linux image: /boot/vmlinuz-5.4.0-187-generic 2024-06-30T12:54:46.756 INFO:teuthology.orchestra.run.smithi167.stderr:Found initrd image: /boot/initrd.img-5.4.0-187-generic 2024-06-30T12:54:46.868 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.4.0-187-generic 2024-06-30T12:54:46.879 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.4.0-187-generic 2024-06-30T12:54:47.166 INFO:teuthology.orchestra.run.smithi167.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-06-30T12:54:47.177 INFO:teuthology.orchestra.run.smithi167.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-06-30T12:54:47.272 INFO:teuthology.orchestra.run.smithi144.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-06-30T12:54:47.284 INFO:teuthology.orchestra.run.smithi144.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-06-30T12:54:47.918 INFO:teuthology.orchestra.run.smithi167.stderr:done 2024-06-30T12:54:47.920 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2024-06-30T12:54:48.064 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-06-30T12:54:48.065 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-06-30T12:55:17.994 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-06-30T12:55:17.995 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-30T12:55:17.996 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T12:55:18.069 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-06-30T12:55:18.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-06-30T12:55:18.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T12:55:36.479 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-06-30T12:55:45.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-30T12:55:45.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T12:55:48.546 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-06-30T12:56:00.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-30T12:56:00.560 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T12:56:18.085 DEBUG:teuthology.orchestra.remote:timed out 2024-06-30T12:56:18.978 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-06-30T12:56:27.092 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-06-30T12:56:27.093 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T12:56:27.499 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-06-30T12:56:28.897 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-06-30T12:56:28.898 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-187-generic"... 2024-06-30T12:56:28.898 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-06-30T12:56:28.904 INFO:teuthology.orchestra.run.smithi144.stdout:5.4.0-187-generic 2024-06-30T12:56:28.905 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-187-generic vs 5.4.0-187-generic 2024-06-30T12:56:28.905 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-30T12:56:28.905 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-06-30T12:56:29.905 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-06-30T12:56:29.906 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-30T12:56:29.990 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-06-30T12:56:30.031 DEBUG:teuthology.parallel:result is None 2024-06-30T12:56:33.979 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-30T12:56:33.980 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T12:56:34.376 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-06-30T12:56:35.274 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-30T12:56:35.274 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-187-generic"... 2024-06-30T12:56:35.274 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-06-30T12:56:35.280 INFO:teuthology.orchestra.run.smithi167.stdout:5.4.0-187-generic 2024-06-30T12:56:35.280 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-187-generic vs 5.4.0-187-generic 2024-06-30T12:56:35.281 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-30T12:56:35.281 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-06-30T12:56:36.281 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-06-30T12:56:36.282 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-30T12:56:36.425 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2024-06-30T12:56:36.462 DEBUG:teuthology.parallel:result is None 2024-06-30T12:56:36.462 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-30T12:56:36.470 INFO:teuthology.task.internal:Creating test directory... 2024-06-30T12:56:36.470 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-30T12:56:36.473 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-30T12:56:36.480 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-30T12:56:36.515 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-30T12:56:36.559 INFO:teuthology.task.internal:Creating archive directory... 2024-06-30T12:56:36.560 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-30T12:56:36.562 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-30T12:56:36.584 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-30T12:56:36.597 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-30T12:56:36.597 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-30T12:56:36.600 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-30T12:56:36.622 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-30T12:56:36.623 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-30T12:56:36.629 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-30T12:56:36.634 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-30T12:56:36.636 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-30T12:56:36.697 INFO:teuthology.task.internal:Configuring sudo... 2024-06-30T12:56:36.697 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-30T12:56:36.700 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-30T12:56:36.715 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-30T12:56:36.735 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-30T12:56:36.735 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-30T12:56:36.738 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-30T12:56:36.762 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-30T12:56:36.791 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-30T12:56:36.797 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T12:56:36.798 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-30T12:56:36.820 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-30T12:56:36.828 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-30T12:56:36.835 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-30T12:56:36.835 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-30T12:56:36.927 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-06-30T12:56:36.929 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-06-30T12:56:37.030 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-30T12:56:37.038 INFO:teuthology.task.internal:Starting timer... 2024-06-30T12:56:37.039 INFO:teuthology.run_tasks:Running task pcp... 2024-06-30T12:56:37.076 INFO:teuthology.run_tasks:Running task selinux... 2024-06-30T12:56:37.108 DEBUG:teuthology.task.selinux:Excluding smithi144: OS 'ubuntu' does not support SELinux 2024-06-30T12:56:37.108 DEBUG:teuthology.task.selinux:Excluding smithi167: OS 'ubuntu' does not support SELinux 2024-06-30T12:56:37.108 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-30T12:56:37.108 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-06-30T12:56:37.108 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-30T12:56:37.108 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-30T12:56:37.145 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-30T12:56:37.158 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-30T12:56:37.159 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi144.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-30T13:02:01.433 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi144.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-06-30T13:02:01.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-06-30T13:02:01.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T13:02:01.528 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-06-30T13:02:01.652 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-06-30T13:02:01.652 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-30T13:02:01.653 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-30T13:02:01.739 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-06-30T13:02:01.864 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-06-30T13:02:01.864 INFO:teuthology.run_tasks:Running task clock... 2024-06-30T13:02:01.875 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-30T13:02:01.875 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-30T13:02:01.876 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-30T13:02:01.878 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-30T13:02:01.878 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-30T13:02:01.906 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-06-30T13:02:01.907 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: Command line: ntpd -gq 2024-06-30T13:02:01.908 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: proto: precision = 0.117 usec (-23) 2024-06-30T13:02:01.909 INFO:teuthology.orchestra.run.smithi144.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-06-30T13:02:01.909 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-06-30T13:02:01.909 INFO:teuthology.orchestra.run.smithi144.stderr:restrict ::: KOD does nothing without LIMITED. 2024-06-30T13:02:01.910 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: restrict ::: KOD does nothing without LIMITED. 2024-06-30T13:02:01.910 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: Listen and drop on 0 v6wildcard [::]:123 2024-06-30T13:02:01.911 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-06-30T13:02:01.911 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: Listen normally on 2 lo 127.0.0.1:123 2024-06-30T13:02:01.911 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: Listen normally on 3 enp3s0f1 172.21.15.144:123 2024-06-30T13:02:01.911 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: Listen normally on 4 lo [::1]:123 2024-06-30T13:02:01.911 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f5f%5]:123 2024-06-30T13:02:01.911 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:01 ntpd[13351]: Listening on routing socket on fd #22 for interface updates 2024-06-30T13:02:01.912 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-06-30T13:02:01.912 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: Command line: ntpd -gq 2024-06-30T13:02:01.913 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: proto: precision = 0.106 usec (-23) 2024-06-30T13:02:01.913 INFO:teuthology.orchestra.run.smithi167.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-06-30T13:02:01.913 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-06-30T13:02:01.914 INFO:teuthology.orchestra.run.smithi167.stderr:restrict ::: KOD does nothing without LIMITED. 2024-06-30T13:02:01.915 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: restrict ::: KOD does nothing without LIMITED. 2024-06-30T13:02:01.915 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: Listen and drop on 0 v6wildcard [::]:123 2024-06-30T13:02:01.915 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-06-30T13:02:01.915 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: Listen normally on 2 lo 127.0.0.1:123 2024-06-30T13:02:01.915 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: Listen normally on 3 enp3s0f1 172.21.15.167:123 2024-06-30T13:02:01.915 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: Listen normally on 4 lo [::1]:123 2024-06-30T13:02:01.915 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7309%5]:123 2024-06-30T13:02:01.916 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:01 ntpd[13349]: Listening on routing socket on fd #22 for interface updates 2024-06-30T13:02:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:02.911 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:02.912 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:02 ntpd[13351]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:04.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:04.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:04.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:04 ntpd[13351]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:04.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:04 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:04.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:04 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:04.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:04 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:04.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:04 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:04.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:04 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:04.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:04 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:04.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:04 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:04.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:04 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:05.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:05 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:05.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:05 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:05.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:05 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:05.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:05 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:05.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:05 ntpd[13351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:05.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:05 ntpd[13351]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:05.912 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:05 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:05.912 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:05 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:05.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:05 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:05.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:05 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:05.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:05 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:05.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:05 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:06.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:06.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:06.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:06.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:06.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:06.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:06.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:06.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:06.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:06.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:06.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:06.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:06 ntpd[13351]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:06.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:06.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:06.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:06 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:07.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:07 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:07.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:07 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:07.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:07 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:07.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:07 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:07.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:07 ntpd[13351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:07.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:07 ntpd[13351]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:07.912 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:07 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:07.912 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:07 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:07.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:07 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:07.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:07 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:07.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:07 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:07.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:07 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:08.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:08.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:08.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:08.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:08.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:08.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:08.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:08.908 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:08.909 INFO:teuthology.orchestra.run.smithi144.stdout:30 Jun 13:02:08 ntpd[13351]: ntpd: time slew +0.003331 s 2024-06-30T13:02:08.909 INFO:teuthology.orchestra.run.smithi144.stdout:ntpd: time slew +0.003331s 2024-06-30T13:02:08.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-06-30T13:02:08.909 INFO:teuthology.orchestra.run.smithi144.stderr:30 Jun 13:02:08 ntpd[13351]: can't open /var/log/ntpstats/loopstats.20240630: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:08.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:08.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:08 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:08.978 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-06-30T13:02:08.978 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-06-30T13:02:08.978 INFO:teuthology.orchestra.run.smithi144.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-06-30T13:02:08.978 INFO:teuthology.orchestra.run.smithi144.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-06-30T13:02:08.978 INFO:teuthology.orchestra.run.smithi144.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-06-30T13:02:08.978 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-06-30T13:02:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:09 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:09 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:09 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:09 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:09 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:09.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:09 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:10.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:10.913 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: can't open /var/log/ntpstats/rawstats.20240630: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-06-30T13:02:10.914 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: can't open /var/log/ntpstats/peerstats.20240630: Permission denied 2024-06-30T13:02:10.915 INFO:teuthology.orchestra.run.smithi167.stdout:30 Jun 13:02:10 ntpd[13349]: ntpd: time slew -0.001605 s 2024-06-30T13:02:10.915 INFO:teuthology.orchestra.run.smithi167.stdout:ntpd: time slew -0.001605s 2024-06-30T13:02:10.915 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-06-30T13:02:10.915 INFO:teuthology.orchestra.run.smithi167.stderr:30 Jun 13:02:10 ntpd[13349]: can't open /var/log/ntpstats/loopstats.20240630: Permission denied 2024-06-30T13:02:10.982 INFO:teuthology.orchestra.run.smithi167.stdout: remote refid st t when poll reach delay offset jitter 2024-06-30T13:02:10.982 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================== 2024-06-30T13:02:10.982 INFO:teuthology.orchestra.run.smithi167.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-06-30T13:02:10.982 INFO:teuthology.orchestra.run.smithi167.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-06-30T13:02:10.982 INFO:teuthology.orchestra.run.smithi167.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-06-30T13:02:10.983 INFO:teuthology.orchestra.run.smithi167.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-06-30T13:02:10.984 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-30T13:02:11.075 INFO:tasks.cephadm:Config: {'allow_ptrace': False, 'avoid_pacific_features': True, 'cephadm_branch': 'v16.2.0', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'image': 'quay.io/ceph/ceph:v16.2.0', 'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'f1abf795f4fc15bf2f30ac90c811a53ab21d5d65'} 2024-06-30T13:02:11.075 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v16.2.0 2024-06-30T13:02:11.075 INFO:tasks.cephadm:Cluster fsid is f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:11.076 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-30T13:02:11.076 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.144', 'mon.c': '[v2:172.21.15.144:3301,v1:172.21.15.144:6790]', 'mon.b': '172.21.15.167'} 2024-06-30T13:02:11.076 INFO:tasks.cephadm:First mon is mon.a on smithi144 2024-06-30T13:02:11.076 INFO:tasks.cephadm:First mgr is y 2024-06-30T13:02:11.076 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-30T13:02:11.076 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-06-30T13:02:11.093 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-06-30T13:02:11.109 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref v16.2.0)... 2024-06-30T13:02:11.109 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-30T13:02:11.413 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-rw-r-- 1 ubuntu ubuntu 288386 Jun 30 13:02 /home/ubuntu/cephtest/cephadm 2024-06-30T13:02:11.414 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.0/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-30T13:02:11.574 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-rw-r-- 1 ubuntu ubuntu 288386 Jun 30 13:02 /home/ubuntu/cephtest/cephadm 2024-06-30T13:02:11.574 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-30T13:02:11.583 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-30T13:02:11.601 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v16.2.0 on all hosts... 2024-06-30T13:02:11.602 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 pull 2024-06-30T13:02:11.604 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 pull 2024-06-30T13:02:11.881 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay.io/ceph/ceph:v16.2.0... 2024-06-30T13:02:11.892 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay.io/ceph/ceph:v16.2.0... 2024-06-30T13:02:43.154 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-06-30T13:02:43.154 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)", 2024-06-30T13:02:43.154 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "24ecd6d5f14c2268c5d2c666aa151456949cf76c6747d873c6fcf41f6f672d15", 2024-06-30T13:02:43.154 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-06-30T13:02:43.154 INFO:teuthology.orchestra.run.smithi144.stdout: "quay.io/ceph/ceph@sha256:9b04c0f15704c49591640a37c7adfd40ffad0a4b42fecb950c3407687cb4f29a" 2024-06-30T13:02:43.154 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-06-30T13:02:43.155 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-06-30T13:02:43.200 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-06-30T13:02:43.200 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)", 2024-06-30T13:02:43.201 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "24ecd6d5f14c2268c5d2c666aa151456949cf76c6747d873c6fcf41f6f672d15", 2024-06-30T13:02:43.201 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-06-30T13:02:43.201 INFO:teuthology.orchestra.run.smithi167.stdout: "quay.io/ceph/ceph@sha256:9b04c0f15704c49591640a37c7adfd40ffad0a4b42fecb950c3407687cb4f29a" 2024-06-30T13:02:43.201 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-06-30T13:02:43.201 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-06-30T13:02:43.225 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-06-30T13:02:43.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-06-30T13:02:43.257 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-06-30T13:02:43.271 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-06-30T13:02:43.284 INFO:tasks.cephadm:Writing seed config... 2024-06-30T13:02:43.286 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-06-30T13:02:43.286 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-30T13:02:43.286 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-30T13:02:43.286 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2024-06-30T13:02:43.286 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-30T13:02:43.286 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-30T13:02:43.286 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-30T13:02:43.286 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-30T13:02:43.286 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-30T13:02:43.287 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T13:02:43.287 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-30T13:02:43.317 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f6ab1df6-36e0-11ef-bcaa-c7b262605968 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-06-30T13:02:43.318 DEBUG:teuthology.orchestra.run.smithi144:mon.a> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a.service 2024-06-30T13:02:43.320 DEBUG:teuthology.orchestra.run.smithi144:mgr.y> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y.service 2024-06-30T13:02:43.322 INFO:tasks.cephadm:Bootstrapping... 2024-06-30T13:02:43.322 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 -v bootstrap --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.144 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-30T13:02:43.474 INFO:teuthology.orchestra.run.smithi144.stderr:Could not locate podman: podman not found 2024-06-30T13:02:43.475 INFO:teuthology.orchestra.run.smithi144.stderr:Verifying podman|docker is present... 2024-06-30T13:02:43.475 INFO:teuthology.orchestra.run.smithi144.stderr:Verifying lvm2 is present... 2024-06-30T13:02:43.475 INFO:teuthology.orchestra.run.smithi144.stderr:Verifying time synchronization is in place... 2024-06-30T13:02:43.475 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled chrony.service 2024-06-30T13:02:43.480 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-06-30T13:02:43.480 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active chrony.service 2024-06-30T13:02:43.488 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: inactive 2024-06-30T13:02:43.488 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled chronyd.service 2024-06-30T13:02:43.494 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Failed to get unit file state for chronyd.service: No such file or directory 2024-06-30T13:02:43.494 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active chronyd.service 2024-06-30T13:02:43.500 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: inactive 2024-06-30T13:02:43.501 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled systemd-timesyncd.service 2024-06-30T13:02:43.507 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: masked 2024-06-30T13:02:43.508 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active systemd-timesyncd.service 2024-06-30T13:02:43.514 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: inactive 2024-06-30T13:02:43.515 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled ntpd.service 2024-06-30T13:02:43.520 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Failed to get unit file state for ntpd.service: No such file or directory 2024-06-30T13:02:43.521 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active ntpd.service 2024-06-30T13:02:43.526 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: inactive 2024-06-30T13:02:43.527 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled ntp.service 2024-06-30T13:02:43.533 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: enabled 2024-06-30T13:02:43.534 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active ntp.service 2024-06-30T13:02:43.539 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: active 2024-06-30T13:02:43.540 INFO:teuthology.orchestra.run.smithi144.stderr:Unit ntp.service is enabled and running 2024-06-30T13:02:43.540 INFO:teuthology.orchestra.run.smithi144.stderr:Repeating the final host check... 2024-06-30T13:02:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:podman|docker (/usr/bin/docker) is present 2024-06-30T13:02:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl is present 2024-06-30T13:02:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:lvcreate is present 2024-06-30T13:02:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled chrony.service 2024-06-30T13:02:43.546 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-06-30T13:02:43.546 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active chrony.service 2024-06-30T13:02:43.552 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: inactive 2024-06-30T13:02:43.553 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled chronyd.service 2024-06-30T13:02:43.559 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Failed to get unit file state for chronyd.service: No such file or directory 2024-06-30T13:02:43.560 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active chronyd.service 2024-06-30T13:02:43.567 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: inactive 2024-06-30T13:02:43.568 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled systemd-timesyncd.service 2024-06-30T13:02:43.574 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: masked 2024-06-30T13:02:43.575 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active systemd-timesyncd.service 2024-06-30T13:02:43.581 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: inactive 2024-06-30T13:02:43.582 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled ntpd.service 2024-06-30T13:02:43.587 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Failed to get unit file state for ntpd.service: No such file or directory 2024-06-30T13:02:43.588 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active ntpd.service 2024-06-30T13:02:43.593 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: inactive 2024-06-30T13:02:43.594 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-enabled ntp.service 2024-06-30T13:02:43.600 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: enabled 2024-06-30T13:02:43.601 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl is-active ntp.service 2024-06-30T13:02:43.606 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: active 2024-06-30T13:02:43.607 INFO:teuthology.orchestra.run.smithi144.stderr:Unit ntp.service is enabled and running 2024-06-30T13:02:43.607 INFO:teuthology.orchestra.run.smithi144.stderr:Host looks OK 2024-06-30T13:02:43.607 INFO:teuthology.orchestra.run.smithi144.stderr:Cluster fsid: f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:43.608 INFO:teuthology.orchestra.run.smithi144.stderr:Acquiring lock 139666891259856 on /run/cephadm/f6ab1df6-36e0-11ef-bcaa-c7b262605968.lock 2024-06-30T13:02:43.608 INFO:teuthology.orchestra.run.smithi144.stderr:Lock 139666891259856 acquired on /run/cephadm/f6ab1df6-36e0-11ef-bcaa-c7b262605968.lock 2024-06-30T13:02:43.608 INFO:teuthology.orchestra.run.smithi144.stderr:Verifying IP 172.21.15.144 port 3300 ... 2024-06-30T13:02:43.609 INFO:teuthology.orchestra.run.smithi144.stderr:Verifying IP 172.21.15.144 port 6789 ... 2024-06-30T13:02:43.609 INFO:teuthology.orchestra.run.smithi144.stderr:Base mon IP is 172.21.15.144, final addrv is [v2:172.21.15.144:3300,v1:172.21.15.144:6789] 2024-06-30T13:02:43.609 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/sbin/ip route ls 2024-06-30T13:02:43.612 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: default via 172.21.15.254 dev enp3s0f1 2024-06-30T13:02:43.612 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.144 metric 100 2024-06-30T13:02:43.612 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2024-06-30T13:02:43.613 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.144 2024-06-30T13:02:43.613 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.144 metric 100 2024-06-30T13:02:43.613 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/sbin/ip -6 route ls 2024-06-30T13:02:43.616 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2024-06-30T13:02:43.616 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-30T13:02:43.616 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 expires 1791sec pref medium 2024-06-30T13:02:43.616 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/sbin/ip -6 addr ls 2024-06-30T13:02:43.620 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-30T13:02:43.620 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: inet6 ::1/128 scope host 2024-06-30T13:02:43.620 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: valid_lft forever preferred_lft forever 2024-06-30T13:02:43.620 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-30T13:02:43.620 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link 2024-06-30T13:02:43.620 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/sbin/ip: valid_lft forever preferred_lft forever 2024-06-30T13:02:43.621 INFO:teuthology.orchestra.run.smithi144.stderr:Mon IP 172.21.15.144 is in CIDR network 172.21.0.0/20 2024-06-30T13:02:43.621 INFO:teuthology.orchestra.run.smithi144.stderr:- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-30T13:02:43.622 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay.io/ceph/ceph:v16.2.0... 2024-06-30T13:02:43.622 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker pull quay.io/ceph/ceph:v16.2.0 2024-06-30T13:02:43.734 INFO:journalctl@ceph.mgr.y.smithi144.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:02:43.748 INFO:journalctl@ceph.mon.a.smithi144.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:02:44.155 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/docker: v16.2.0: Pulling from ceph/ceph 2024-06-30T13:02:44.270 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/docker: Digest: sha256:9b04c0f15704c49591640a37c7adfd40ffad0a4b42fecb950c3407687cb4f29a 2024-06-30T13:02:44.270 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/docker: Status: Image is up to date for quay.io/ceph/ceph:v16.2.0 2024-06-30T13:02:44.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/docker: quay.io/ceph/ceph:v16.2.0 2024-06-30T13:02:44.306 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 quay.io/ceph/ceph:v16.2.0 --version 2024-06-30T13:02:45.439 INFO:teuthology.orchestra.run.smithi144.stderr:ceph: ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable) 2024-06-30T13:02:45.945 INFO:teuthology.orchestra.run.smithi144.stderr:Ceph version: ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable) 2024-06-30T13:02:45.945 INFO:teuthology.orchestra.run.smithi144.stderr:Extracting ceph user uid/gid from container image... 2024-06-30T13:02:45.945 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint stat --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 quay.io/ceph/ceph:v16.2.0 -c %u %g /var/lib/ceph 2024-06-30T13:02:46.850 INFO:teuthology.orchestra.run.smithi144.stderr:stat: 167 167 2024-06-30T13:02:47.458 INFO:teuthology.orchestra.run.smithi144.stderr:Creating initial keys... 2024-06-30T13:02:47.458 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 quay.io/ceph/ceph:v16.2.0 --gen-print-key 2024-06-30T13:02:48.404 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-authtool: AQB4V4Fm7Rn2FxAAzIDm17+ooBjXBDqmI7byeA== 2024-06-30T13:02:49.006 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 quay.io/ceph/ceph:v16.2.0 --gen-print-key 2024-06-30T13:02:49.946 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-authtool: AQB5V4Fmu9xqOBAA5wLTsZ9KY8X/vVZB0e1UJQ== 2024-06-30T13:02:50.554 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 quay.io/ceph/ceph:v16.2.0 --gen-print-key 2024-06-30T13:02:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-authtool: AQB7V4FmIsXNGBAAKnjG8kLT8v0ECmGveaFZxw== 2024-06-30T13:02:51.979 INFO:teuthology.orchestra.run.smithi144.stderr:Creating initial monmap... 2024-06-30T13:02:51.979 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/monmaptool --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /tmp/ceph-tmpw7chh4ak:/tmp/monmap:z quay.io/ceph/ceph:v16.2.0 --create --clobber --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 --addv a [v2:172.21.15.144:3300,v1:172.21.15.144:6789] /tmp/monmap 2024-06-30T13:02:52.974 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-30T13:02:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:52.975 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-30T13:02:53.508 INFO:teuthology.orchestra.run.smithi144.stderr:monmaptool for a [v2:172.21.15.144:3300,v1:172.21.15.144:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-30T13:02:53.509 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/monmaptool: set fsid to f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:53.509 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-30T13:02:53.509 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-06-30T13:02:53.509 INFO:teuthology.orchestra.run.smithi144.stderr:Creating mon... 2024-06-30T13:02:53.511 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph-mon --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a:/var/lib/ceph/mon/ceph-a:z -v /tmp/ceph-tmpqcopx4y_:/tmp/keyring:z -v /tmp/ceph-tmpw7chh4ak:/tmp/monmap:z quay.io/ceph/ceph:v16.2.0 --mkfs -i a --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -c /dev/null --monmap /tmp/monmap --keyring /tmp/keyring --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true 2024-06-30T13:02:54.504 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.498+0000 7fa0dc6a4700 0 set uid:gid to 167:167 (ceph:ceph) 2024-06-30T13:02:54.504 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.498+0000 7fa0dc6a4700 1 imported monmap: 2024-06-30T13:02:54.504 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: epoch 0 2024-06-30T13:02:54.505 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:54.505 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: last_changed 2024-06-30T13:02:52.974804+0000 2024-06-30T13:02:54.505 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: created 2024-06-30T13:02:52.974804+0000 2024-06-30T13:02:54.505 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-06-30T13:02:54.506 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-06-30T13:02:54.506 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.a 2024-06-30T13:02:54.507 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.507 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.498+0000 7fa0dc6a4700 0 /usr/bin/ceph-mon: set fsid to f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:54.507 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: RocksDB version: 6.8.1 2024-06-30T13:02:54.507 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.507 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-06-30T13:02:54.508 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Compile date Mar 30 2021 2024-06-30T13:02:54.508 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: DB SUMMARY 2024-06-30T13:02:54.513 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.513 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-06-30T13:02:54.513 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.513 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-06-30T13:02:54.513 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.513 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.error_if_exists: 0 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.create_if_missing: 1 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.paranoid_checks: 1 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.env: 0x55ae6bcf31c0 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.fs: Posix File System 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.info_log: 0x55ae6d390f40 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_file_opening_threads: 16 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.statistics: (nil) 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.use_fsync: 0 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_log_file_size: 0 2024-06-30T13:02:54.514 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.keep_log_file_num: 1000 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.recycle_log_file_num: 0 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.allow_fallocate: 1 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.allow_mmap_reads: 0 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.allow_mmap_writes: 0 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.use_direct_reads: 0 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.create_missing_column_families: 0 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.db_log_dir: 2024-06-30T13:02:54.515 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_subcompactions: 1 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_background_flushes: -1 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.advise_random_on_open: 1 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.db_write_buffer_size: 0 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.write_buffer_manager: 0x55ae6d39b410 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-30T13:02:54.516 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-06-30T13:02:54.517 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.rate_limiter: (nil) 2024-06-30T13:02:54.517 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-30T13:02:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.wal_recovery_mode: 2 2024-06-30T13:02:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.enable_thread_tracking: 0 2024-06-30T13:02:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.enable_pipelined_write: 0 2024-06-30T13:02:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.unordered_write: 0 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.row_cache: None 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.wal_filter: None 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.allow_ingest_behind: 0 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.preserve_deletes: 0 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.two_write_queues: 0 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.manual_wal_flush: 0 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.atomic_flush: 0 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-30T13:02:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.log_readahead_size: 0 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_background_jobs: 2 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_background_compactions: -1 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_total_wal_size: 0 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-06-30T13:02:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_open_files: -1 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.bytes_per_sync: 0 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_readahead_size: 0 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Compression algorithms supported: 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: kZSTD supported: 0 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: kXpressCompression supported: 0 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: kLZ4HCCompression supported: 1 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: kLZ4Compression supported: 1 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: kBZip2Compression supported: 0 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: kZlibCompression supported: 1 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: kSnappyCompression supported: 1 2024-06-30T13:02:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.merge_operator: 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_filter: None 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_filter_factory: None 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ae6d2a80d0) 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-06-30T13:02:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: index_type: 0 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: checksum: 1 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: block_cache: 0x55ae6d2e0d10 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-06-30T13:02:54.541 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-06-30T13:02:54.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: format_version: 2 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: block_align: 0 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.write_buffer_size: 33554432 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_write_buffer_number: 2 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compression: NoCompression 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.bottommost_compression: Disabled 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.prefix_extractor: nullptr 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.num_levels: 7 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-30T13:02:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compression_opts.level: 32767 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compression_opts.strategy: 0 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compression_opts.enabled: false 2024-06-30T13:02:54.544 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.target_file_size_base: 67108864 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-30T13:02:54.545 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.arena_block_size: 4194304 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.disable_auto_compactions: 0 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-30T13:02:54.546 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.table_properties_collectors: 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.inplace_update_support: 0 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.bloom_locality: 0 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.max_successive_merges: 0 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.paranoid_file_checks: 0 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.force_consistency_checks: 0 2024-06-30T13:02:54.547 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.report_bg_io_stats: 0 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.ttl: 2592000 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.502+0000 7fa0dc6a4700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.510+0000 7fa0dc6a4700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.510+0000 7fa0dc6a4700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.510+0000 7fa0dc6a4700 4 rocksdb: DB pointer 0x55ae6d3a5800 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.510+0000 7fa0c553f700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.510+0000 7fa0c553f700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-30T13:02:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:02:54.549 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:02:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-30T13:02:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.552 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-06-30T13:02:54.552 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: 2024-06-30T13:02:54.552 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.514+0000 7fa0dc6a4700 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-06-30T13:02:54.552 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.514+0000 7fa0dc6a4700 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-06-30T13:02:54.552 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph-mon: debug 2024-06-30T13:02:54.514+0000 7fa0dc6a4700 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-06-30T13:02:55.064 INFO:teuthology.orchestra.run.smithi144.stderr:create mon.a on 2024-06-30T13:02:55.065 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: install -d -m0770 -o 167 -g 167 /var/run/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:55.073 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl disable ceph.target 2024-06-30T13:02:55.288 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl enable ceph.target 2024-06-30T13:02:55.295 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2024-06-30T13:02:55.479 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl start ceph.target 2024-06-30T13:02:55.487 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl enable ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968.target 2024-06-30T13:02:55.494 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968.target -> /etc/systemd/system/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968.target. 2024-06-30T13:02:55.494 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968.target -> /etc/systemd/system/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968.target. 2024-06-30T13:02:55.680 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl start ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968.target 2024-06-30T13:02:55.689 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl daemon-reload 2024-06-30T13:02:55.885 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a 2024-06-30T13:02:55.899 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl reset-failed ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a 2024-06-30T13:02:55.905 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Failed to reset failed state of unit ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a.service: Unit ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a.service not loaded. 2024-06-30T13:02:55.906 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl enable ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a 2024-06-30T13:02:55.912 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Created symlink /etc/systemd/system/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968.target.wants/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a.service -> /etc/systemd/system/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@.service. 2024-06-30T13:02:56.109 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl start ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a 2024-06-30T13:02:56.122 INFO:teuthology.orchestra.run.smithi144.stderr:firewalld does not appear to be present 2024-06-30T13:02:56.122 INFO:teuthology.orchestra.run.smithi144.stderr:Not possible to enable service . firewalld.service is not available 2024-06-30T13:02:56.122 INFO:teuthology.orchestra.run.smithi144.stderr:Not possible to open ports <[]>. firewalld.service is not available 2024-06-30T13:02:56.123 INFO:teuthology.orchestra.run.smithi144.stderr:Waiting for mon to start... 2024-06-30T13:02:56.123 INFO:teuthology.orchestra.run.smithi144.stderr:Waiting for mon... 2024-06-30T13:02:56.123 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a:/var/lib/ceph/mon/ceph-a:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 status 2024-06-30T13:02:56.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:56 smithi144 systemd[1]: Started Ceph mon.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:02:57.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:57 smithi144 bash[14308]: cluster 2024-06-30T13:02:57.626813+0000 mon.a (mon.0) 0 : [INF] mkfs f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:57.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:57 smithi144 bash[14308]: cluster 2024-06-30T13:02:57.614257+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-06-30T13:02:57.864 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: cluster: 2024-06-30T13:02:57.865 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: id: f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:57.865 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-06-30T13:02:57.865 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:02:57.865 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: services: 2024-06-30T13:02:57.866 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.240658s) 2024-06-30T13:02:57.866 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mgr: no daemons active 2024-06-30T13:02:57.866 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-06-30T13:02:57.866 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:02:57.867 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: data: 2024-06-30T13:02:57.867 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-06-30T13:02:57.867 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-06-30T13:02:57.867 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-06-30T13:02:57.867 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: pgs: 2024-06-30T13:02:57.867 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:02:58.519 INFO:teuthology.orchestra.run.smithi144.stderr:mon is available 2024-06-30T13:02:58.519 INFO:teuthology.orchestra.run.smithi144.stderr:Assimilating anything we can from ceph.conf... 2024-06-30T13:02:58.519 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z -v /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a:/var/lib/ceph/mon/ceph-a:z quay.io/ceph/ceph:v16.2.0 config assimilate-conf -i /var/lib/ceph/mon/ceph-a/config 2024-06-30T13:02:59.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:58 smithi144 bash[14308]: cluster 2024-06-30T13:02:57.623585+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-06-30T13:02:59.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:58 smithi144 bash[14308]: cluster 2024-06-30T13:02:57.625653+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} 2024-06-30T13:02:59.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:58 smithi144 bash[14308]: cluster 2024-06-30T13:02:57.629904+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2024-06-30T13:02:59.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:58 smithi144 bash[14308]: cluster 2024-06-30T13:02:57.634533+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-06-30T13:02:59.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:58 smithi144 bash[14308]: cluster 2024-06-30T13:02:57.635121+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-06-30T13:02:59.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:58 smithi144 bash[14308]: audit 2024-06-30T13:02:57.864178+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.144:0/702170352' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-30T13:02:59.727 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:02:59.727 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: [global] 2024-06-30T13:02:59.727 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: fsid = f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:02:59.728 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: log_to_journald = false 2024-06-30T13:02:59.728 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.144:3300,v1:172.21.15.144:6789] 2024-06-30T13:02:59.729 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: [mgr] 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mgr/cephadm/use_agent = True 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: [mon] 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mon_warn_on_insecure_global_id_reclaim = false 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:02:59.730 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: [osd] 2024-06-30T13:02:59.731 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-06-30T13:02:59.731 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: osd_memory_target_autotune = true 2024-06-30T13:02:59.731 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-06-30T13:03:00.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:59 smithi144 bash[14308]: audit 2024-06-30T13:02:59.721239+0000 mon.a (mon.0) 8 : audit [INF] from='client.? 172.21.15.144:0/3779375346' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-30T13:03:00.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:02:59 smithi144 bash[14308]: audit 2024-06-30T13:02:59.726484+0000 mon.a (mon.0) 9 : audit [INF] from='client.? 172.21.15.144:0/3779375346' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-30T13:03:00.325 INFO:teuthology.orchestra.run.smithi144.stderr:Generating new minimal ceph.conf... 2024-06-30T13:03:00.325 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z -v /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a:/var/lib/ceph/mon/ceph-a:z quay.io/ceph/ceph:v16.2.0 config generate-minimal-conf -o /var/lib/ceph/mon/ceph-a/config 2024-06-30T13:03:02.006 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:01 smithi144 bash[14308]: audit 2024-06-30T13:03:01.491032+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.144:0/980144265' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:03:02.065 INFO:teuthology.orchestra.run.smithi144.stderr:Restarting the monitor... 2024-06-30T13:03:02.066 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl restart ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a 2024-06-30T13:03:02.436 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:02 smithi144 systemd[1]: Stopping Ceph mon.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:03:02.436 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:02 smithi144 bash[14308]: debug 2024-06-30T13:03:02.166+0000 7fdf1801e700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-06-30T13:03:02.437 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:02 smithi144 bash[14308]: debug 2024-06-30T13:03:02.166+0000 7fdf1801e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-30T13:03:03.008 INFO:teuthology.orchestra.run.smithi144.stderr:Setting mon public_network to 172.21.0.0/20 2024-06-30T13:03:03.008 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 config set mon public_network 172.21.0.0/20 2024-06-30T13:03:03.140 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:02 smithi144 docker[14702]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-mon.a 2024-06-30T13:03:03.140 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:03 smithi144 systemd[1]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a.service: Succeeded. 2024-06-30T13:03:03.140 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:03 smithi144 systemd[1]: Stopped Ceph mon.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:03:03.140 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:03 smithi144 systemd[1]: Started Ceph mon.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:03:04.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.289+0000 7fa6c1557700 0 set uid:gid to 167:167 (ceph:ceph) 2024-06-30T13:03:04.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.289+0000 7fa6c1557700 0 ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable), process ceph-mon, pid 7 2024-06-30T13:03:04.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.289+0000 7fa6c1557700 0 pidfile_write: ignore empty --pid-file 2024-06-30T13:03:04.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.293+0000 7fa6c1557700 0 load: jerasure load: lrc load: isa 2024-06-30T13:03:04.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: RocksDB version: 6.8.1 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Compile date Mar 30 2021 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: DB SUMMARY 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: CURRENT file: CURRENT 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: IDENTITY file: IDENTITY 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 83108 ; 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.error_if_exists: 0 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.create_if_missing: 0 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.paranoid_checks: 1 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.env: 0x55a107e051c0 2024-06-30T13:03:04.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.fs: Posix File System 2024-06-30T13:03:04.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.info_log: 0x55a10942afe0 2024-06-30T13:03:04.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_file_opening_threads: 16 2024-06-30T13:03:04.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.statistics: (nil) 2024-06-30T13:03:04.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.use_fsync: 0 2024-06-30T13:03:04.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_log_file_size: 0 2024-06-30T13:03:04.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-30T13:03:04.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-06-30T13:03:04.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.keep_log_file_num: 1000 2024-06-30T13:03:04.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.recycle_log_file_num: 0 2024-06-30T13:03:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.allow_fallocate: 1 2024-06-30T13:03:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.allow_mmap_reads: 0 2024-06-30T13:03:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.allow_mmap_writes: 0 2024-06-30T13:03:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.use_direct_reads: 0 2024-06-30T13:03:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-30T13:03:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.create_missing_column_families: 0 2024-06-30T13:03:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.db_log_dir: 2024-06-30T13:03:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_subcompactions: 1 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_background_flushes: -1 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.advise_random_on_open: 1 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.db_write_buffer_size: 0 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.write_buffer_manager: 0x55a1094c1e30 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-06-30T13:03:04.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.rate_limiter: (nil) 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.wal_recovery_mode: 2 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.enable_thread_tracking: 0 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.enable_pipelined_write: 0 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.unordered_write: 0 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.row_cache: None 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.wal_filter: None 2024-06-30T13:03:04.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.allow_ingest_behind: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.preserve_deletes: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.two_write_queues: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.manual_wal_flush: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.atomic_flush: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.log_readahead_size: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_background_jobs: 2 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_background_compactions: -1 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-30T13:03:04.513 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_total_wal_size: 0 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_open_files: -1 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.bytes_per_sync: 0 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_readahead_size: 0 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Compression algorithms supported: 2024-06-30T13:03:04.514 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: kZSTD supported: 0 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: kXpressCompression supported: 0 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: kLZ4HCCompression supported: 1 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: kLZ4Compression supported: 1 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: kBZip2Compression supported: 0 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: kZlibCompression supported: 1 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: kSnappyCompression supported: 1 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.merge_operator: 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_filter: None 2024-06-30T13:03:04.515 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_filter_factory: None 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a1093ce0d8) 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: cache_index_and_filter_blocks: 1 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: pin_top_level_index_and_filter: 1 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: index_type: 0 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: data_block_index_type: 0 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: index_shortening: 1 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: data_block_hash_table_util_ratio: 0.750000 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: hash_index_allow_collision: 1 2024-06-30T13:03:04.516 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: checksum: 1 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: no_block_cache: 0 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: block_cache: 0x55a109406f10 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: block_cache_name: BinnedLRUCache 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: block_cache_options: 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: capacity : 536870912 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: num_shard_bits : 4 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: strict_capacity_limit : 0 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: high_pri_pool_ratio: 0.000 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: block_cache_compressed: (nil) 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: persistent_cache: (nil) 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: block_size: 4096 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: block_size_deviation: 10 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: block_restart_interval: 16 2024-06-30T13:03:04.517 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: index_block_restart_interval: 1 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: metadata_block_size: 4096 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: partition_filters: 0 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: use_delta_encoding: 1 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: filter_policy: rocksdb.BuiltinBloomFilter 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: whole_key_filtering: 1 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: verify_compression: 0 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: read_amp_bytes_per_bit: 0 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: format_version: 2 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: enable_index_compression: 1 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: block_align: 0 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.write_buffer_size: 33554432 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_write_buffer_number: 2 2024-06-30T13:03:04.518 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compression: NoCompression 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.bottommost_compression: Disabled 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.prefix_extractor: nullptr 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.num_levels: 7 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-06-30T13:03:04.519 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compression_opts.level: 32767 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compression_opts.strategy: 0 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compression_opts.enabled: false 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.target_file_size_base: 67108864 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-30T13:03:04.520 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-30T13:03:04.521 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-30T13:03:04.521 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-30T13:03:04.521 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-30T13:03:04.521 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-30T13:03:04.521 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.arena_block_size: 4194304 2024-06-30T13:03:04.521 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.disable_auto_compactions: 0 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-30T13:03:04.522 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.table_properties_collectors: 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.inplace_update_support: 0 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.bloom_locality: 0 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.max_successive_merges: 0 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.paranoid_file_checks: 0 2024-06-30T13:03:04.523 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.force_consistency_checks: 0 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.report_bg_io_stats: 0 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.ttl: 2592000 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1719752584302971, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1719752584304240, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 80076, "table_properties": {"data_size": 78334, "index_size": 347, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 581, "raw_key_size": 8979, "raw_average_key_size": 46, "raw_value_size": 73093, "raw_average_value_size": 376, "num_data_blocks": 11, "num_entries": 194, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1719752584, "oldest_key_time": 3, "file_creation_time": 0}} 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.297+0000 7fa6c1557700 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.301+0000 7fa6c1557700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1719752584305554, "job": 1, "event": "recovery_finished"} 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.301+0000 7fa6c1557700 4 rocksdb: DB pointer 0x55a1094cb800 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.301+0000 7fa6a8371700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-06-30T13:03:04.524 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.301+0000 7fa6a8371700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ** DB Stats ** 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Uptime(secs): 0.0 total, 0.0 interval 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ** Compaction Stats [default] ** 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: L0 2/0 79.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.1 0.00 0.00 1 0.001 0 0 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Sum 2/0 79.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.1 0.00 0.00 1 0.001 0 0 2024-06-30T13:03:04.525 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.1 0.00 0.00 1 0.001 0 0 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ** Compaction Stats [default] ** 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 78.1 0.00 0.00 1 0.001 0 0 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Uptime(secs): 0.0 total, 0.0 interval 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Flush(GB): cumulative 0.000, interval 0.000 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: AddFile(Total Files): cumulative 0, interval 0 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: AddFile(L0 Files): cumulative 0, interval 0 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: AddFile(Keys): cumulative 0, interval 0 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Cumulative compaction: 0.00 GB write, 13.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Interval compaction: 0.00 GB write, 13.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:03:04.526 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ** File Read Latency Histogram By Level [default] ** 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ** Compaction Stats [default] ** 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: L0 2/0 79.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.1 0.00 0.00 1 0.001 0 0 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Sum 2/0 79.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 78.1 0.00 0.00 1 0.001 0 0 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ** Compaction Stats [default] ** 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 78.1 0.00 0.00 1 0.001 0 0 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Uptime(secs): 0.0 total, 0.0 interval 2024-06-30T13:03:04.527 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Flush(GB): cumulative 0.000, interval 0.000 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: AddFile(Total Files): cumulative 0, interval 0 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: AddFile(L0 Files): cumulative 0, interval 0 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: AddFile(Keys): cumulative 0, interval 0 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Cumulative compaction: 0.00 GB write, 13.80 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: ** File Read Latency Histogram By Level [default] ** 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.301+0000 7fa6c1557700 0 starting mon.a rank 0 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 1 mon.a@-1(???) e1 preinit fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 0 mon.a@-1(???).mds e1 new map 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 0 mon.a@-1(???).mds e1 print_map 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: e1 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: enable_multiple, ever_enabled_multiple: 1,1 2024-06-30T13:03:04.528 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: legacy client fscid: -1 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: No filesystems configured 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: audit 2024-06-30T13:03:01.491032+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? 172.21.15.144:0/980144265' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta expand map: {default=false} 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta from 'false' to 'false' 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta expanded map: {default=false} 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta expand map: {default=info} 2024-06-30T13:03:04.529 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta from 'info' to 'info' 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta expanded map: {default=info} 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta expand map: {default=daemon} 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta from 'daemon' to 'daemon' 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta expanded map: {default=daemon} 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta expand map: {default=debug} 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta from 'debug' to 'debug' 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: debug 2024-06-30T13:03:04.305+0000 7fa6c1557700 20 expand_channel_meta expanded map: {default=debug} 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: cluster 2024-06-30T13:03:04.316365+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: cluster 2024-06-30T13:03:04.316456+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: cluster 2024-06-30T13:03:04.316529+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: cluster 2024-06-30T13:03:04.316576+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-06-30T13:03:04.530 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:04 smithi144 bash[14758]: cluster 2024-06-30T13:03:04.317073+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-06-30T13:03:05.195 INFO:teuthology.orchestra.run.smithi144.stderr:Wrote config to /etc/ceph/ceph.conf 2024-06-30T13:03:05.195 INFO:teuthology.orchestra.run.smithi144.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-30T13:03:05.195 INFO:teuthology.orchestra.run.smithi144.stderr:Creating mgr... 2024-06-30T13:03:05.196 INFO:teuthology.orchestra.run.smithi144.stderr:Verifying port 9283 ... 2024-06-30T13:03:05.198 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl daemon-reload 2024-06-30T13:03:05.409 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y 2024-06-30T13:03:05.420 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl reset-failed ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y 2024-06-30T13:03:05.425 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Failed to reset failed state of unit ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y.service: Unit ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y.service not loaded. 2024-06-30T13:03:05.425 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl enable ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y 2024-06-30T13:03:05.431 INFO:teuthology.orchestra.run.smithi144.stderr:systemctl: Created symlink /etc/systemd/system/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968.target.wants/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y.service -> /etc/systemd/system/ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@.service. 2024-06-30T13:03:05.623 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: systemctl start ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y 2024-06-30T13:03:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:firewalld does not appear to be present 2024-06-30T13:03:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:Not possible to enable service . firewalld.service is not available 2024-06-30T13:03:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:Not possible to open ports <[]>. firewalld.service is not available 2024-06-30T13:03:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:firewalld does not appear to be present 2024-06-30T13:03:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-06-30T13:03:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:Waiting for mgr to start... 2024-06-30T13:03:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:Waiting for mgr... 2024-06-30T13:03:05.633 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 status -f json-pretty 2024-06-30T13:03:06.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:05 smithi144 bash[14758]: audit 2024-06-30T13:03:04.588166+0000 mon.a (mon.0) 6 : audit [INF] from='client.? 172.21.15.144:0/3770404204' entity='client.admin' 2024-06-30T13:03:07.300 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:03:07.300 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: { 2024-06-30T13:03:07.300 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "fsid": "f6ab1df6-36e0-11ef-bcaa-c7b262605968", 2024-06-30T13:03:07.300 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "health": { 2024-06-30T13:03:07.300 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-06-30T13:03:07.301 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "checks": {}, 2024-06-30T13:03:07.301 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "mutes": [] 2024-06-30T13:03:07.301 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:07.301 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-06-30T13:03:07.301 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "quorum": [ 2024-06-30T13:03:07.301 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 0 2024-06-30T13:03:07.301 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ], 2024-06-30T13:03:07.301 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "quorum_names": [ 2024-06-30T13:03:07.302 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "a" 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ], 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "quorum_age": 2, 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "monmap": { 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_mons": 1 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "osdmap": { 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_osds": 0, 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-06-30T13:03:07.303 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "pgmap": { 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_pools": 0, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_objects": 0, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-06-30T13:03:07.307 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "bytes_total": 0 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "fsmap": { 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "by_rank": [], 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "up:standby": 0 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "mgrmap": { 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "available": false, 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "modules": [ 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "iostat", 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "restful" 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ], 2024-06-30T13:03:07.308 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "services": {} 2024-06-30T13:03:07.309 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:07.309 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "servicemap": { 2024-06-30T13:03:07.309 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:07.309 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "modified": "2024-06-30T13:02:57.627515+0000", 2024-06-30T13:03:07.309 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "services": {} 2024-06-30T13:03:07.309 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:07.309 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "progress_events": {} 2024-06-30T13:03:07.309 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: } 2024-06-30T13:03:07.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:07 smithi144 bash[14758]: audit 2024-06-30T13:03:07.298949+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? 172.21.15.144:0/861336376' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-30T13:03:07.920 INFO:teuthology.orchestra.run.smithi144.stderr:mgr not available, waiting (1/15)... 2024-06-30T13:03:09.923 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 status -f json-pretty 2024-06-30T13:03:11.188 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:03:11.189 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: { 2024-06-30T13:03:11.189 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "fsid": "f6ab1df6-36e0-11ef-bcaa-c7b262605968", 2024-06-30T13:03:11.189 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "health": { 2024-06-30T13:03:11.189 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-06-30T13:03:11.193 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "checks": {}, 2024-06-30T13:03:11.193 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "mutes": [] 2024-06-30T13:03:11.193 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:11.193 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-06-30T13:03:11.193 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "quorum": [ 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 0 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ], 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "quorum_names": [ 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "a" 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ], 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "quorum_age": 6, 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "monmap": { 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_mons": 1 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:11.194 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "osdmap": { 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_osds": 0, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "pgmap": { 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_pools": 0, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_objects": 0, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-06-30T13:03:11.195 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "bytes_total": 0 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "fsmap": { 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "by_rank": [], 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "up:standby": 0 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "mgrmap": { 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "available": false, 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "modules": [ 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "iostat", 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "restful" 2024-06-30T13:03:11.196 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ], 2024-06-30T13:03:11.197 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "services": {} 2024-06-30T13:03:11.197 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:11.197 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "servicemap": { 2024-06-30T13:03:11.197 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:11.197 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "modified": "2024-06-30T13:02:57.627515+0000", 2024-06-30T13:03:11.197 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "services": {} 2024-06-30T13:03:11.197 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:11.197 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "progress_events": {} 2024-06-30T13:03:11.197 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: } 2024-06-30T13:03:11.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:11 smithi144 bash[14758]: audit 2024-06-30T13:03:11.187960+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? 172.21.15.144:0/2509281036' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-30T13:03:11.794 INFO:teuthology.orchestra.run.smithi144.stderr:mgr not available, waiting (2/15)... 2024-06-30T13:03:13.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: cluster 2024-06-30T13:03:12.863102+0000 mon.a (mon.0) 9 : cluster [INF] Activating manager daemon y 2024-06-30T13:03:13.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: cluster 2024-06-30T13:03:12.869024+0000 mon.a (mon.0) 10 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00606874s) 2024-06-30T13:03:13.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.871711+0000 mon.a (mon.0) 11 : audit [DBG] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-30T13:03:13.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.872251+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-30T13:03:13.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.872637+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-30T13:03:13.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.872989+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-30T13:03:13.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.873394+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-30T13:03:13.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: cluster 2024-06-30T13:03:12.876168+0000 mon.a (mon.0) 16 : cluster [INF] Manager daemon y is now available 2024-06-30T13:03:13.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.891905+0000 mon.a (mon.0) 17 : audit [INF] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' 2024-06-30T13:03:13.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.895112+0000 mon.a (mon.0) 18 : audit [INF] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:03:13.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.896936+0000 mon.a (mon.0) 19 : audit [INF] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:03:13.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.904961+0000 mon.a (mon.0) 20 : audit [INF] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' 2024-06-30T13:03:13.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:12 smithi144 bash[14758]: audit 2024-06-30T13:03:12.910815+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 172.21.15.144:0/3838238714' entity='mgr.y' 2024-06-30T13:03:13.797 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 status -f json-pretty 2024-06-30T13:03:15.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:14 smithi144 bash[14758]: cluster 2024-06-30T13:03:13.874872+0000 mon.a (mon.0) 22 : cluster [DBG] mgrmap e3: y(active, since 1.01194s) 2024-06-30T13:03:15.341 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 2024-06-30T13:03:15.341 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: { 2024-06-30T13:03:15.341 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "fsid": "f6ab1df6-36e0-11ef-bcaa-c7b262605968", 2024-06-30T13:03:15.341 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "health": { 2024-06-30T13:03:15.341 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-06-30T13:03:15.341 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "checks": {}, 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "mutes": [] 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "quorum": [ 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 0 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ], 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "quorum_names": [ 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "a" 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ], 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "quorum_age": 11, 2024-06-30T13:03:15.342 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "monmap": { 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_mons": 1 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "osdmap": { 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_osds": 0, 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-06-30T13:03:15.343 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "pgmap": { 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_pools": 0, 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_objects": 0, 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "bytes_total": 0 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "fsmap": { 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "by_rank": [], 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "up:standby": 0 2024-06-30T13:03:15.344 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "mgrmap": { 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "available": true, 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "modules": [ 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "iostat", 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "restful" 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ], 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "services": {} 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "servicemap": { 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 1, 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "modified": "2024-06-30T13:02:57.627515+0000", 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "services": {} 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: }, 2024-06-30T13:03:15.345 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "progress_events": {} 2024-06-30T13:03:15.346 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: } 2024-06-30T13:03:15.869 INFO:teuthology.orchestra.run.smithi144.stderr:mgr is available 2024-06-30T13:03:15.870 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 config-key set mgr/cephadm/ssh_user root 2024-06-30T13:03:16.736 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:16 smithi144 bash[14758]: cluster 2024-06-30T13:03:15.324361+0000 mon.a (mon.0) 23 : cluster [DBG] mgrmap e4: y(active, since 2s) 2024-06-30T13:03:16.736 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:16 smithi144 bash[14758]: audit 2024-06-30T13:03:15.332121+0000 mon.a (mon.0) 24 : audit [DBG] from='client.? 172.21.15.144:0/1995637157' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-30T13:03:17.452 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: set mgr/cephadm/ssh_user 2024-06-30T13:03:18.068 INFO:teuthology.orchestra.run.smithi144.stderr:Enabling cephadm module... 2024-06-30T13:03:18.068 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 mgr module enable cephadm 2024-06-30T13:03:18.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:18 smithi144 bash[14758]: audit 2024-06-30T13:03:17.447743+0000 mon.a (mon.0) 25 : audit [INF] from='client.? 172.21.15.144:0/2658275916' entity='client.admin' 2024-06-30T13:03:20.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:20 smithi144 bash[14758]: audit 2024-06-30T13:03:19.597497+0000 mon.a (mon.0) 26 : audit [INF] from='client.? 172.21.15.144:0/2300346972' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-30T13:03:20.757 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:20 smithi144 bash[15024]: ignoring --setgroup ceph since I am not root 2024-06-30T13:03:20.961 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 mgr stat 2024-06-30T13:03:21.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:21 smithi144 bash[14758]: audit 2024-06-30T13:03:20.329418+0000 mon.a (mon.0) 27 : audit [INF] from='client.? 172.21.15.144:0/2300346972' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-30T13:03:21.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:21 smithi144 bash[14758]: cluster 2024-06-30T13:03:20.329541+0000 mon.a (mon.0) 28 : cluster [DBG] mgrmap e5: y(active, since 7s) 2024-06-30T13:03:22.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: { 2024-06-30T13:03:22.539 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 5, 2024-06-30T13:03:22.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "available": true, 2024-06-30T13:03:22.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "active_name": "y", 2024-06-30T13:03:22.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_standby": 0 2024-06-30T13:03:22.540 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: } 2024-06-30T13:03:23.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:22 smithi144 bash[14758]: audit 2024-06-30T13:03:22.538706+0000 mon.a (mon.0) 29 : audit [DBG] from='client.? 172.21.15.144:0/4001170642' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-30T13:03:23.151 INFO:teuthology.orchestra.run.smithi144.stderr:Waiting for the mgr to restart... 2024-06-30T13:03:23.151 INFO:teuthology.orchestra.run.smithi144.stderr:Waiting for mgr epoch 5... 2024-06-30T13:03:23.151 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 tell mgr mgr_status 2024-06-30T13:03:26.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:26 smithi144 bash[14758]: cluster 2024-06-30T13:03:26.190989+0000 mon.a (mon.0) 30 : cluster [INF] Active manager daemon y restarted 2024-06-30T13:03:26.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:26 smithi144 bash[14758]: cluster 2024-06-30T13:03:26.191695+0000 mon.a (mon.0) 31 : cluster [INF] Activating manager daemon y 2024-06-30T13:03:26.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:26 smithi144 bash[14758]: cluster 2024-06-30T13:03:26.197295+0000 mon.a (mon.0) 32 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-06-30T13:03:27.274 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: { 2024-06-30T13:03:27.274 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-06-30T13:03:27.274 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "initialized": true 2024-06-30T13:03:27.274 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: } 2024-06-30T13:03:27.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: cluster 2024-06-30T13:03:26.245386+0000 mon.a (mon.0) 33 : cluster [DBG] mgrmap e6: y(active, starting, since 0.0538221s) 2024-06-30T13:03:27.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.250188+0000 mon.a (mon.0) 34 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-30T13:03:27.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.251867+0000 mon.a (mon.0) 35 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-30T13:03:27.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.253212+0000 mon.a (mon.0) 36 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-30T13:03:27.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.253635+0000 mon.a (mon.0) 37 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-30T13:03:27.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.254025+0000 mon.a (mon.0) 38 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-30T13:03:27.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: cluster 2024-06-30T13:03:26.258361+0000 mon.a (mon.0) 39 : cluster [INF] Manager daemon y is now available 2024-06-30T13:03:27.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.282589+0000 mon.a (mon.0) 40 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:27.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.288611+0000 mon.a (mon.0) 41 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:27.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.294588+0000 mon.a (mon.0) 42 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:27.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.421846+0000 mon.a (mon.0) 43 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:27.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.423083+0000 mon.a (mon.0) 44 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:03:27.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.426811+0000 mon.a (mon.0) 45 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:03:27.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.431467+0000 mon.a (mon.0) 46 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:03:27.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.446106+0000 mon.a (mon.0) 47 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:03:27.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:27 smithi144 bash[14758]: audit 2024-06-30T13:03:26.448243+0000 mon.a (mon.0) 48 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:03:27.972 INFO:teuthology.orchestra.run.smithi144.stderr:mgr epoch 5 is available 2024-06-30T13:03:27.972 INFO:teuthology.orchestra.run.smithi144.stderr:Setting orchestrator backend to cephadm... 2024-06-30T13:03:27.972 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 orch set backend cephadm 2024-06-30T13:03:28.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:28 smithi144 bash[14758]: audit 2024-06-30T13:03:27.259645+0000 mgr.y (mgr.14116) 1 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-30T13:03:28.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:28 smithi144 bash[14758]: cluster 2024-06-30T13:03:27.259733+0000 mon.a (mon.0) 49 : cluster [DBG] mgrmap e7: y(active, since 1.06817s) 2024-06-30T13:03:28.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:28 smithi144 bash[14758]: audit 2024-06-30T13:03:27.273673+0000 mgr.y (mgr.14116) 2 : audit [DBG] from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-30T13:03:29.555 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:29 smithi144 bash[14758]: cluster 2024-06-30T13:03:28.265969+0000 mon.a (mon.0) 50 : cluster [DBG] mgrmap e8: y(active, since 2s) 2024-06-30T13:03:30.079 INFO:teuthology.orchestra.run.smithi144.stderr:Generating ssh key... 2024-06-30T13:03:30.080 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 cephadm generate-key 2024-06-30T13:03:30.968 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:30 smithi144 bash[14758]: audit 2024-06-30T13:03:29.516713+0000 mgr.y (mgr.14116) 3 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:03:30.968 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:30 smithi144 bash[14758]: audit 2024-06-30T13:03:29.526025+0000 mon.a (mon.0) 51 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:30.968 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:30 smithi144 bash[14758]: audit 2024-06-30T13:03:29.534765+0000 mon.a (mon.0) 52 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:03:32.007 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: Generating public/private rsa key pair. 2024-06-30T13:03:32.007 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: Your identification has been saved in /tmp/tmppkza33xa/key. 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: Your public key has been saved in /tmp/tmppkza33xa/key.pub. 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: The key fingerprint is: 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: SHA256:UgT+t0RHSA/9SDXSHTTDwfVvyryRCyeALdLpaCAU9sk ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: The key's randomart image is: 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: +---[RSA 3072]----+ 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: | o ....oooo*B=| 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: | . + o . .+o..+=| 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: | . E . . ..oo .| 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: | . + = .. . .| 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: | . . o S = o| 2024-06-30T13:03:32.008 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: | . . = + o o + | 2024-06-30T13:03:32.009 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: | o . . o B | 2024-06-30T13:03:32.009 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: | . + + | 2024-06-30T13:03:32.009 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: | o | 2024-06-30T13:03:32.009 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:31 smithi144 bash[15024]: +----[SHA256]-----+ 2024-06-30T13:03:32.256 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 cephadm get-pub-key 2024-06-30T13:03:33.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:32 smithi144 bash[14758]: cephadm 2024-06-30T13:03:31.495513+0000 mgr.y (mgr.14116) 5 : cephadm [INF] Generating ssh key... 2024-06-30T13:03:33.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:32 smithi144 bash[14758]: audit 2024-06-30T13:03:31.678577+0000 mon.a (mon.0) 53 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:33.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:32 smithi144 bash[14758]: audit 2024-06-30T13:03:31.683742+0000 mon.a (mon.0) 54 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:33.736 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ1hZ6kJQq99TFj5yW4jRMM6FoFVtsrAuNj4+bQQmmAknJbcvN+ab5COOiM2p86kKB8uTB+0P4K+oDu0HTuTwHYqdOSlTNg+KvEYS8krraIuahUcUZxbHD57mlFfMPtLm7MZQpRot3xk0YsJbrSd+u3R7QWXxjzN+Kf3h+3ocVXHz/pqHVbdKpvWFwlMQXgmuwrtCDHK1Wl85nblTi501xopKgscEl8kPliJgWvHbRfJzYkgTPXnB3GtiLgCe01QSQ8BnM+yogLAVN2y0ffgViPVYV2HFtkEVZOrdK0DArdogWHWXcQlceN8pplqoVb1h2bLLjjWO3fYTN1vo4bscyH6TZ5gMszq4VZNbKJaf9u2ora9kZh9K3lsYLhn04YNAtwDUBVAm4gqZwDP4kKasqHoERKYkFT0MDVeV7g7CEF6gOL0rYWQw7vGbKIksOiMQCsgSepcpheK8+YV1ym3rcFKXH99L0jUjRxfQ6CVkINSVG2BLRH+m2CZNG39sWhIc= ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:03:34.296 INFO:teuthology.orchestra.run.smithi144.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-30T13:03:34.296 INFO:teuthology.orchestra.run.smithi144.stderr:Adding key to root@localhost authorized_keys... 2024-06-30T13:03:34.297 INFO:teuthology.orchestra.run.smithi144.stderr:Adding host smithi144... 2024-06-30T13:03:34.297 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 orch host add smithi144 2024-06-30T13:03:35.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:35 smithi144 bash[14758]: audit 2024-06-30T13:03:33.735128+0000 mgr.y (mgr.14116) 6 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:03:36.007 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:35 smithi144 bash[15024]: Warning: Permanently added 'smithi144,172.21.15.144' (ECDSA) to the list of known hosts. 2024-06-30T13:03:36.506 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: Added host 'smithi144' 2024-06-30T13:03:36.677 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:36 smithi144 bash[14758]: audit 2024-06-30T13:03:35.714990+0000 mgr.y (mgr.14116) 7 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:03:37.344 INFO:teuthology.orchestra.run.smithi144.stderr:Deploying unmanaged mon service... 2024-06-30T13:03:37.345 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 orch apply mon --unmanaged 2024-06-30T13:03:37.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:37 smithi144 bash[14758]: audit 2024-06-30T13:03:36.504362+0000 mon.a (mon.0) 55 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:37.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:37 smithi144 bash[14758]: cephadm 2024-06-30T13:03:36.505083+0000 mgr.y (mgr.14116) 8 : cephadm [INF] Added host smithi144 2024-06-30T13:03:37.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:37 smithi144 bash[14758]: audit 2024-06-30T13:03:36.505607+0000 mon.a (mon.0) 56 : audit [DBG] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:03:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: Scheduled mon update... 2024-06-30T13:03:40.674 INFO:teuthology.orchestra.run.smithi144.stderr:Deploying unmanaged mgr service... 2024-06-30T13:03:40.674 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 orch apply mgr --unmanaged 2024-06-30T13:03:41.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:40 smithi144 bash[14758]: audit 2024-06-30T13:03:39.661709+0000 mgr.y (mgr.14116) 9 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:03:41.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:40 smithi144 bash[14758]: cephadm 2024-06-30T13:03:39.663725+0000 mgr.y (mgr.14116) 10 : cephadm [INF] Saving service mon spec with placement count:5 2024-06-30T13:03:41.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:40 smithi144 bash[14758]: audit 2024-06-30T13:03:39.666619+0000 mon.a (mon.0) 57 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:41.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:40 smithi144 bash[14758]: audit 2024-06-30T13:03:40.286327+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:41.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:40 smithi144 bash[14758]: audit 2024-06-30T13:03:40.532367+0000 mon.a (mon.0) 59 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:42.264 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-06-30T13:03:42.841 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 config set mgr mgr/cephadm/container_init True --force 2024-06-30T13:03:43.745 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:43 smithi144 bash[14758]: audit 2024-06-30T13:03:42.258755+0000 mgr.y (mgr.14116) 11 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:03:43.746 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:43 smithi144 bash[14758]: cephadm 2024-06-30T13:03:42.259892+0000 mgr.y (mgr.14116) 12 : cephadm [INF] Saving service mgr spec with placement count:2 2024-06-30T13:03:43.746 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:43 smithi144 bash[14758]: audit 2024-06-30T13:03:42.262797+0000 mon.a (mon.0) 60 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:45.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:44 smithi144 bash[14758]: audit 2024-06-30T13:03:43.657905+0000 mon.a (mon.0) 61 : audit [INF] from='mgr.14116 172.21.15.144:0/3424753456' entity='mgr.y' 2024-06-30T13:03:45.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:44 smithi144 bash[14758]: audit 2024-06-30T13:03:44.289441+0000 mon.a (mon.0) 62 : audit [INF] from='client.? 172.21.15.144:0/220846284' entity='client.admin' 2024-06-30T13:03:45.404 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 config set mgr mgr/dashboard/ssl_server_port 8443 2024-06-30T13:03:47.583 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:47 smithi144 bash[14758]: cluster 2024-06-30T13:03:46.254818+0000 mgr.y (mgr.14116) 13 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:03:48.183 INFO:teuthology.orchestra.run.smithi144.stderr:Enabling the dashboard module... 2024-06-30T13:03:48.184 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 mgr module enable dashboard 2024-06-30T13:03:49.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:48 smithi144 bash[14758]: audit 2024-06-30T13:03:47.551992+0000 mon.a (mon.0) 63 : audit [INF] from='client.? 172.21.15.144:0/2719413915' entity='client.admin' 2024-06-30T13:03:49.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:48 smithi144 bash[14758]: cluster 2024-06-30T13:03:48.255308+0000 mgr.y (mgr.14116) 14 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:03:50.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:50 smithi144 bash[14758]: audit 2024-06-30T13:03:49.795483+0000 mon.a (mon.0) 64 : audit [INF] from='client.? 172.21.15.144:0/2193070860' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-30T13:03:50.757 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:50 smithi144 bash[15024]: ignoring --setuser ceph since I am not root 2024-06-30T13:03:50.757 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:50 smithi144 bash[15024]: ignoring --setgroup ceph since I am not root 2024-06-30T13:03:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 mgr stat 2024-06-30T13:03:51.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:51 smithi144 bash[14758]: audit 2024-06-30T13:03:50.336183+0000 mon.a (mon.0) 65 : audit [INF] from='client.? 172.21.15.144:0/2193070860' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-30T13:03:51.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:51 smithi144 bash[14758]: cluster 2024-06-30T13:03:50.336296+0000 mon.a (mon.0) 66 : cluster [DBG] mgrmap e9: y(active, since 24s) 2024-06-30T13:03:52.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: { 2024-06-30T13:03:52.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "epoch": 9, 2024-06-30T13:03:52.542 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "available": true, 2024-06-30T13:03:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "active_name": "y", 2024-06-30T13:03:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "num_standby": 0 2024-06-30T13:03:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: } 2024-06-30T13:03:53.007 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:52 smithi144 bash[14758]: audit 2024-06-30T13:03:52.541401+0000 mon.a (mon.0) 67 : audit [DBG] from='client.? 172.21.15.144:0/3284289825' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-30T13:03:53.197 INFO:teuthology.orchestra.run.smithi144.stderr:Waiting for the mgr to restart... 2024-06-30T13:03:53.197 INFO:teuthology.orchestra.run.smithi144.stderr:Waiting for mgr epoch 9... 2024-06-30T13:03:53.197 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 tell mgr mgr_status 2024-06-30T13:03:56.553 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:03:56 smithi144 bash[15024]: Warning: Permanently added 'smithi144,172.21.15.144' (ECDSA) to the list of known hosts. 2024-06-30T13:03:56.553 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:56 smithi144 bash[14758]: cluster 2024-06-30T13:03:56.235004+0000 mon.a (mon.0) 68 : cluster [INF] Active manager daemon y restarted 2024-06-30T13:03:56.553 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:56 smithi144 bash[14758]: cluster 2024-06-30T13:03:56.235711+0000 mon.a (mon.0) 69 : cluster [INF] Activating manager daemon y 2024-06-30T13:03:56.553 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:56 smithi144 bash[14758]: cluster 2024-06-30T13:03:56.238134+0000 mon.a (mon.0) 70 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-06-30T13:03:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: { 2024-06-30T13:03:57.318 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-06-30T13:03:57.318 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: "initialized": true 2024-06-30T13:03:57.318 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: } 2024-06-30T13:03:57.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: cluster 2024-06-30T13:03:56.291568+0000 mon.a (mon.0) 71 : cluster [DBG] mgrmap e10: y(active, starting, since 0.0559541s) 2024-06-30T13:03:57.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: audit 2024-06-30T13:03:56.297517+0000 mon.a (mon.0) 72 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-30T13:03:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: audit 2024-06-30T13:03:56.299210+0000 mon.a (mon.0) 73 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-30T13:03:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: audit 2024-06-30T13:03:56.300427+0000 mon.a (mon.0) 74 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-30T13:03:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: audit 2024-06-30T13:03:56.300722+0000 mon.a (mon.0) 75 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-30T13:03:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: audit 2024-06-30T13:03:56.300932+0000 mon.a (mon.0) 76 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-30T13:03:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: cluster 2024-06-30T13:03:56.305562+0000 mon.a (mon.0) 77 : cluster [INF] Manager daemon y is now available 2024-06-30T13:03:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: audit 2024-06-30T13:03:56.454524+0000 mon.a (mon.0) 78 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:03:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: audit 2024-06-30T13:03:56.461149+0000 mon.a (mon.0) 79 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:03:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: audit 2024-06-30T13:03:56.495774+0000 mon.a (mon.0) 80 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:03:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:57 smithi144 bash[14758]: audit 2024-06-30T13:03:56.498704+0000 mon.a (mon.0) 81 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:03:57.999 INFO:teuthology.orchestra.run.smithi144.stderr:mgr epoch 9 is available 2024-06-30T13:03:58.000 INFO:teuthology.orchestra.run.smithi144.stderr:Generating a dashboard self-signed certificate... 2024-06-30T13:03:58.000 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 dashboard create-self-signed-cert 2024-06-30T13:03:58.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:58 smithi144 bash[14758]: audit 2024-06-30T13:03:57.301400+0000 mgr.y (mgr.14144) 1 : audit [DBG] from='client.14148 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-30T13:03:58.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:58 smithi144 bash[14758]: cluster 2024-06-30T13:03:57.302413+0000 mon.a (mon.0) 82 : cluster [DBG] mgrmap e11: y(active, since 1.06679s) 2024-06-30T13:03:58.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:58 smithi144 bash[14758]: audit 2024-06-30T13:03:57.317249+0000 mgr.y (mgr.14144) 2 : audit [DBG] from='client.14148 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-30T13:03:59.502 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: Self-signed certificate created 2024-06-30T13:03:59.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:03:59 smithi144 bash[14758]: cluster 2024-06-30T13:03:58.317277+0000 mon.a (mon.0) 83 : cluster [DBG] mgrmap e12: y(active, since 2s) 2024-06-30T13:04:00.364 INFO:teuthology.orchestra.run.smithi144.stderr:Creating initial admin user... 2024-06-30T13:04:00.365 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpmfy3s0b4:/tmp/dashboard.pw:z quay.io/ceph/ceph:v16.2.0 dashboard ac-user-create admin -i /tmp/dashboard.pw administrator --force-password --pwd-update-required 2024-06-30T13:04:00.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:00 smithi144 bash[14758]: audit 2024-06-30T13:03:59.450468+0000 mgr.y (mgr.14144) 3 : audit [DBG] from='client.14154 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:04:00.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:00 smithi144 bash[14758]: audit 2024-06-30T13:03:59.497059+0000 mon.a (mon.0) 84 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:00.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:00 smithi144 bash[14758]: audit 2024-06-30T13:03:59.501594+0000 mon.a (mon.0) 85 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:00.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:00 smithi144 bash[14758]: audit 2024-06-30T13:03:59.774584+0000 mon.a (mon.0) 86 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:02.747 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$SxIcNZmOk11R0./ba.NT3.4pVHHd65CtRPg0gOmZOfHn.oRVM2oOu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1719752642, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-30T13:04:03.361 INFO:teuthology.orchestra.run.smithi144.stderr:Fetching dashboard port number... 2024-06-30T13:04:03.362 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=quay.io/ceph/ceph:v16.2.0 -e NODE_NAME=smithi144 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968:/var/log/ceph:z -v /tmp/ceph-tmpz155u6ho:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmp40p6qp7v:/etc/ceph/ceph.conf:z quay.io/ceph/ceph:v16.2.0 config get mgr mgr/dashboard/ssl_server_port 2024-06-30T13:04:03.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:03 smithi144 bash[14758]: cluster 2024-06-30T13:04:02.294750+0000 mon.a (mon.0) 87 : cluster [DBG] mgrmap e13: y(active, since 6s) 2024-06-30T13:04:03.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:03 smithi144 bash[14758]: audit 2024-06-30T13:04:02.746414+0000 mon.a (mon.0) 88 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:04.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:04 smithi144 bash[14758]: audit 2024-06-30T13:04:02.502240+0000 mgr.y (mgr.14144) 4 : audit [DBG] from='client.14156 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:04:04.879 INFO:teuthology.orchestra.run.smithi144.stderr:/usr/bin/ceph: 8443 2024-06-30T13:04:05.477 INFO:teuthology.orchestra.run.smithi144.stderr:firewalld does not appear to be present 2024-06-30T13:04:05.477 INFO:teuthology.orchestra.run.smithi144.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-30T13:04:05.482 INFO:teuthology.orchestra.run.smithi144.stderr:Ceph Dashboard is now available at: 2024-06-30T13:04:05.482 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-06-30T13:04:05.482 INFO:teuthology.orchestra.run.smithi144.stderr: URL: https://smithi144.front.sepia.ceph.com:8443/ 2024-06-30T13:04:05.482 INFO:teuthology.orchestra.run.smithi144.stderr: User: admin 2024-06-30T13:04:05.482 INFO:teuthology.orchestra.run.smithi144.stderr: Password: 5bebqrtipr 2024-06-30T13:04:05.482 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr:You can access the Ceph CLI with: 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr:Please consider enabling telemetry to help improve Ceph: 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr: ceph telemetry on 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr:For more information see: 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr: https://docs.ceph.com/docs/pacific/mgr/telemetry/ 2024-06-30T13:04:05.483 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-06-30T13:04:05.484 INFO:teuthology.orchestra.run.smithi144.stderr:Bootstrap complete. 2024-06-30T13:04:05.484 INFO:teuthology.orchestra.run.smithi144.stderr:Releasing lock 139666891259856 on /run/cephadm/f6ab1df6-36e0-11ef-bcaa-c7b262605968.lock 2024-06-30T13:04:05.484 INFO:teuthology.orchestra.run.smithi144.stderr:Lock 139666891259856 released on /run/cephadm/f6ab1df6-36e0-11ef-bcaa-c7b262605968.lock 2024-06-30T13:04:05.506 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:05 smithi144 bash[14758]: audit 2024-06-30T13:04:04.878280+0000 mon.a (mon.0) 89 : audit [DBG] from='client.? 172.21.15.144:0/2475077172' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-30T13:04:05.519 INFO:tasks.cephadm:Fetching config... 2024-06-30T13:04:05.519 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T13:04:05.519 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-30T13:04:05.524 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-30T13:04:05.524 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T13:04:05.524 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-30T13:04:05.529 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-30T13:04:05.529 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T13:04:05.529 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-30T13:04:05.590 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-30T13:04:05.590 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T13:04:05.590 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-30T13:04:05.597 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-30T13:04:05.597 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ1hZ6kJQq99TFj5yW4jRMM6FoFVtsrAuNj4+bQQmmAknJbcvN+ab5COOiM2p86kKB8uTB+0P4K+oDu0HTuTwHYqdOSlTNg+KvEYS8krraIuahUcUZxbHD57mlFfMPtLm7MZQpRot3xk0YsJbrSd+u3R7QWXxjzN+Kf3h+3ocVXHz/pqHVbdKpvWFwlMQXgmuwrtCDHK1Wl85nblTi501xopKgscEl8kPliJgWvHbRfJzYkgTPXnB3GtiLgCe01QSQ8BnM+yogLAVN2y0ffgViPVYV2HFtkEVZOrdK0DArdogWHWXcQlceN8pplqoVb1h2bLLjjWO3fYTN1vo4bscyH6TZ5gMszq4VZNbKJaf9u2ora9kZh9K3lsYLhn04YNAtwDUBVAm4gqZwDP4kKasqHoERKYkFT0MDVeV7g7CEF6gOL0rYWQw7vGbKIksOiMQCsgSepcpheK8+YV1ym3rcFKXH99L0jUjRxfQ6CVkINSVG2BLRH+m2CZNG39sWhIc= ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-30T13:04:05.618 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ1hZ6kJQq99TFj5yW4jRMM6FoFVtsrAuNj4+bQQmmAknJbcvN+ab5COOiM2p86kKB8uTB+0P4K+oDu0HTuTwHYqdOSlTNg+KvEYS8krraIuahUcUZxbHD57mlFfMPtLm7MZQpRot3xk0YsJbrSd+u3R7QWXxjzN+Kf3h+3ocVXHz/pqHVbdKpvWFwlMQXgmuwrtCDHK1Wl85nblTi501xopKgscEl8kPliJgWvHbRfJzYkgTPXnB3GtiLgCe01QSQ8BnM+yogLAVN2y0ffgViPVYV2HFtkEVZOrdK0DArdogWHWXcQlceN8pplqoVb1h2bLLjjWO3fYTN1vo4bscyH6TZ5gMszq4VZNbKJaf9u2ora9kZh9K3lsYLhn04YNAtwDUBVAm4gqZwDP4kKasqHoERKYkFT0MDVeV7g7CEF6gOL0rYWQw7vGbKIksOiMQCsgSepcpheK8+YV1ym3rcFKXH99L0jUjRxfQ6CVkINSVG2BLRH+m2CZNG39sWhIc= ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:04:05.636 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ1hZ6kJQq99TFj5yW4jRMM6FoFVtsrAuNj4+bQQmmAknJbcvN+ab5COOiM2p86kKB8uTB+0P4K+oDu0HTuTwHYqdOSlTNg+KvEYS8krraIuahUcUZxbHD57mlFfMPtLm7MZQpRot3xk0YsJbrSd+u3R7QWXxjzN+Kf3h+3ocVXHz/pqHVbdKpvWFwlMQXgmuwrtCDHK1Wl85nblTi501xopKgscEl8kPliJgWvHbRfJzYkgTPXnB3GtiLgCe01QSQ8BnM+yogLAVN2y0ffgViPVYV2HFtkEVZOrdK0DArdogWHWXcQlceN8pplqoVb1h2bLLjjWO3fYTN1vo4bscyH6TZ5gMszq4VZNbKJaf9u2ora9kZh9K3lsYLhn04YNAtwDUBVAm4gqZwDP4kKasqHoERKYkFT0MDVeV7g7CEF6gOL0rYWQw7vGbKIksOiMQCsgSepcpheK8+YV1ym3rcFKXH99L0jUjRxfQ6CVkINSVG2BLRH+m2CZNG39sWhIc= ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-30T13:04:05.664 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ1hZ6kJQq99TFj5yW4jRMM6FoFVtsrAuNj4+bQQmmAknJbcvN+ab5COOiM2p86kKB8uTB+0P4K+oDu0HTuTwHYqdOSlTNg+KvEYS8krraIuahUcUZxbHD57mlFfMPtLm7MZQpRot3xk0YsJbrSd+u3R7QWXxjzN+Kf3h+3ocVXHz/pqHVbdKpvWFwlMQXgmuwrtCDHK1Wl85nblTi501xopKgscEl8kPliJgWvHbRfJzYkgTPXnB3GtiLgCe01QSQ8BnM+yogLAVN2y0ffgViPVYV2HFtkEVZOrdK0DArdogWHWXcQlceN8pplqoVb1h2bLLjjWO3fYTN1vo4bscyH6TZ5gMszq4VZNbKJaf9u2ora9kZh9K3lsYLhn04YNAtwDUBVAm4gqZwDP4kKasqHoERKYkFT0MDVeV7g7CEF6gOL0rYWQw7vGbKIksOiMQCsgSepcpheK8+YV1ym3rcFKXH99L0jUjRxfQ6CVkINSVG2BLRH+m2CZNG39sWhIc= ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:04:05.674 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-06-30T13:04:05.674 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-30T13:04:05.674 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-06-30T13:04:05.723 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-30T13:04:05.723 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-30T13:04:05.729 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-06-30T13:04:05.730 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch host add smithi167 2024-06-30T13:04:10.757 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:04:10 smithi144 bash[15024]: Warning: Permanently added 'smithi167,172.21.15.167' (ECDSA) to the list of known hosts. 2024-06-30T13:04:11.220 INFO:teuthology.orchestra.run.smithi144.stdout:Added host 'smithi167' 2024-06-30T13:04:11.429 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:11 smithi144 bash[14758]: audit 2024-06-30T13:04:10.101335+0000 mon.a (mon.0) 90 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:11.430 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:11 smithi144 bash[14758]: audit 2024-06-30T13:04:10.473817+0000 mon.a (mon.0) 91 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:11.430 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:11 smithi144 bash[14758]: audit 2024-06-30T13:04:10.475367+0000 mon.a (mon.0) 92 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:11.430 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:11 smithi144 bash[14758]: audit 2024-06-30T13:04:10.479737+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:11.912 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch host ls --format=json 2024-06-30T13:04:12.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:12 smithi144 bash[14758]: audit 2024-06-30T13:04:11.218538+0000 mon.a (mon.0) 94 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:12.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:12 smithi144 bash[14758]: cephadm 2024-06-30T13:04:11.219318+0000 mgr.y (mgr.14144) 6 : cephadm [INF] Added host smithi167 2024-06-30T13:04:12.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:12 smithi144 bash[14758]: audit 2024-06-30T13:04:11.219779+0000 mon.a (mon.0) 95 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:12.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:12 smithi144 bash[14758]: audit 2024-06-30T13:04:11.471368+0000 mon.a (mon.0) 96 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:12.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:12 smithi144 bash[14758]: audit 2024-06-30T13:04:12.187236+0000 mon.a (mon.0) 97 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:15.924 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:04:15.924 INFO:teuthology.orchestra.run.smithi144.stdout:[{"addr": "smithi144", "hostname": "smithi144", "labels": [], "status": ""}, {"addr": "smithi167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-06-30T13:04:16.581 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-30T13:04:16.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd crush tunables default 2024-06-30T13:04:17.742 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:17 smithi144 bash[14758]: audit 2024-06-30T13:04:15.923432+0000 mgr.y (mgr.14144) 7 : audit [DBG] from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-30T13:04:17.742 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:17 smithi144 bash[14758]: cluster 2024-06-30T13:04:16.302040+0000 mgr.y (mgr.14144) 8 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:17.742 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:17 smithi144 bash[14758]: audit 2024-06-30T13:04:16.894838+0000 mon.a (mon.0) 98 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:17.742 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:17 smithi144 bash[14758]: audit 2024-06-30T13:04:17.173766+0000 mon.a (mon.0) 99 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:17.742 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:17 smithi144 bash[14758]: audit 2024-06-30T13:04:17.179148+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:19.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:19 smithi144 bash[14758]: cluster 2024-06-30T13:04:18.302452+0000 mgr.y (mgr.14144) 9 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:20.372 INFO:teuthology.orchestra.run.smithi144.stderr:adjusted tunables profile to default 2024-06-30T13:04:20.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:20 smithi144 bash[14758]: audit 2024-06-30T13:04:19.827254+0000 mon.a (mon.0) 101 : audit [INF] from='client.? 172.21.15.144:0/876118173' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-30T13:04:21.047 INFO:tasks.cephadm:Adding mon.a on smithi144 2024-06-30T13:04:21.047 INFO:tasks.cephadm:Adding mon.c on smithi144 2024-06-30T13:04:21.047 INFO:tasks.cephadm:Adding mon.b on smithi167 2024-06-30T13:04:21.047 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch apply mon '3;smithi144:172.21.15.144=a;smithi144:[v2:172.21.15.144:3301,v1:172.21.15.144:6790]=c;smithi167:172.21.15.167=b' 2024-06-30T13:04:21.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:21 smithi144 bash[14758]: cluster 2024-06-30T13:04:20.302862+0000 mgr.y (mgr.14144) 10 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:21.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:21 smithi144 bash[14758]: audit 2024-06-30T13:04:20.367567+0000 mon.a (mon.0) 102 : audit [INF] from='client.? 172.21.15.144:0/876118173' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-30T13:04:21.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:21 smithi144 bash[14758]: cluster 2024-06-30T13:04:20.367704+0000 mon.a (mon.0) 103 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-06-30T13:04:23.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:23 smithi144 bash[14758]: cluster 2024-06-30T13:04:22.303299+0000 mgr.y (mgr.14144) 11 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:24.042 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-06-30T13:04:24.798 DEBUG:teuthology.orchestra.run.smithi144:mon.c> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.c.service 2024-06-30T13:04:24.801 DEBUG:teuthology.orchestra.run.smithi167:mon.b> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.b.service 2024-06-30T13:04:24.803 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-30T13:04:24.803 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph mon dump -f json 2024-06-30T13:04:25.157 INFO:journalctl@ceph.mon.c.smithi144.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:04:25.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:25 smithi144 bash[14758]: audit 2024-06-30T13:04:24.033123+0000 mgr.y (mgr.14144) 12 : audit [DBG] from='client.14166 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi144:172.21.15.144=a;smithi144:[v2:172.21.15.144:3301,v1:172.21.15.144:6790]=c;smithi167:172.21.15.167=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:04:25.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:25 smithi144 bash[14758]: cephadm 2024-06-30T13:04:24.036420+0000 mgr.y (mgr.14144) 13 : cephadm [INF] Saving service mon spec with placement smithi144:172.21.15.144=a;smithi144:[v2:172.21.15.144:3301,v1:172.21.15.144:6790]=c;smithi167:172.21.15.167=b;count:3 2024-06-30T13:04:25.257 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:25 smithi144 bash[14758]: audit 2024-06-30T13:04:24.041417+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:25.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:25 smithi144 bash[14758]: audit 2024-06-30T13:04:24.042742+0000 mon.a (mon.0) 105 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:25.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:25 smithi144 bash[14758]: audit 2024-06-30T13:04:24.052382+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:25.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:25 smithi144 bash[14758]: audit 2024-06-30T13:04:24.054758+0000 mon.a (mon.0) 107 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:25.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:25 smithi144 bash[14758]: audit 2024-06-30T13:04:24.056718+0000 mon.a (mon.0) 108 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:25.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:25 smithi144 bash[14758]: cephadm 2024-06-30T13:04:24.058506+0000 mgr.y (mgr.14144) 14 : cephadm [INF] Deploying daemon mon.b on smithi167 2024-06-30T13:04:25.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:25 smithi144 bash[14758]: cluster 2024-06-30T13:04:24.303707+0000 mgr.y (mgr.14144) 15 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:26.404 INFO:journalctl@ceph.mon.b.smithi167.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:04:27.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:27 smithi144 bash[14758]: cluster 2024-06-30T13:04:26.304133+0000 mgr.y (mgr.14144) 16 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:29.640 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-30T13:04:29.640 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"f6ab1df6-36e0-11ef-bcaa-c7b262605968","modified":"2024-06-30T13:02:52.974804Z","created":"2024-06-30T13:02:52.974804Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-30T13:04:29.643 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-06-30T13:04:29.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:29 smithi144 bash[14758]: cluster 2024-06-30T13:04:28.304548+0000 mgr.y (mgr.14144) 17 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:30.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:30 smithi144 bash[14758]: audit 2024-06-30T13:04:29.639674+0000 mon.a (mon.0) 109 : audit [DBG] from='client.? 172.21.15.167:0/2922750599' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-30T13:04:30.918 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:30 smithi167 systemd[1]: Started Ceph mon.b for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:04:31.253 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-30T13:04:31.253 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph mon dump -f json 2024-06-30T13:04:31.507 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:31 smithi144 bash[14758]: cluster 2024-06-30T13:04:30.304987+0000 mgr.y (mgr.14144) 18 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:31.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:31 smithi144 bash[14758]: audit 2024-06-30T13:04:30.812217+0000 mon.a (mon.0) 110 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:31.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:31 smithi144 bash[14758]: audit 2024-06-30T13:04:30.813628+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:31.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:31 smithi144 bash[14758]: audit 2024-06-30T13:04:30.815392+0000 mon.a (mon.0) 112 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:32.644 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.463+0000 7f0eeb545700 0 set uid:gid to 167:167 (ceph:ceph) 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.463+0000 7f0eeb545700 0 ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable), process ceph-mon, pid 7 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.463+0000 7f0eeb545700 0 pidfile_write: ignore empty --pid-file 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 0 load: jerasure load: lrc load: isa 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: RocksDB version: 6.8.1 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Compile date Mar 30 2021 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: DB SUMMARY 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: CURRENT file: CURRENT 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: IDENTITY file: IDENTITY 2024-06-30T13:04:32.645 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-06-30T13:04:32.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-06-30T13:04:32.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000003.log size: 511 ; 2024-06-30T13:04:32.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.error_if_exists: 0 2024-06-30T13:04:32.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.create_if_missing: 0 2024-06-30T13:04:32.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.paranoid_checks: 1 2024-06-30T13:04:32.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.env: 0x55f0c434d1c0 2024-06-30T13:04:32.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.fs: Posix File System 2024-06-30T13:04:32.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.info_log: 0x55f0c67f2fe0 2024-06-30T13:04:32.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_file_opening_threads: 16 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.statistics: (nil) 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.use_fsync: 0 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_log_file_size: 0 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.log_file_time_to_roll: 0 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.keep_log_file_num: 1000 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.recycle_log_file_num: 0 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.allow_fallocate: 1 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.allow_mmap_reads: 0 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.allow_mmap_writes: 0 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.use_direct_reads: 0 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.create_missing_column_families: 0 2024-06-30T13:04:32.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.db_log_dir: 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.table_cache_numshardbits: 6 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_subcompactions: 1 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_background_flushes: -1 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.advise_random_on_open: 1 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.db_write_buffer_size: 0 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.write_buffer_manager: 0x55f0c6889e30 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-06-30T13:04:32.648 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.use_adaptive_mutex: 0 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.rate_limiter: (nil) 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.wal_recovery_mode: 2 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.enable_thread_tracking: 0 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.enable_pipelined_write: 0 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.unordered_write: 0 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.row_cache: None 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.wal_filter: None 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.allow_ingest_behind: 0 2024-06-30T13:04:32.649 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.preserve_deletes: 0 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.two_write_queues: 0 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.manual_wal_flush: 0 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.atomic_flush: 0 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.persist_stats_to_disk: 0 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.log_readahead_size: 0 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_background_jobs: 2 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_background_compactions: -1 2024-06-30T13:04:32.650 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-30T13:04:32.651 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-30T13:04:32.651 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.delayed_write_rate : 16777216 2024-06-30T13:04:32.651 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_total_wal_size: 0 2024-06-30T13:04:32.651 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-30T13:04:32.651 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.stats_dump_period_sec: 600 2024-06-30T13:04:32.651 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.stats_persist_period_sec: 600 2024-06-30T13:04:32.652 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-30T13:04:32.652 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_open_files: -1 2024-06-30T13:04:32.652 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.bytes_per_sync: 0 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_readahead_size: 0 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Compression algorithms supported: 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: kZSTD supported: 0 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: kXpressCompression supported: 0 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: kLZ4HCCompression supported: 1 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: kLZ4Compression supported: 1 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: kBZip2Compression supported: 0 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: kZlibCompression supported: 1 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: kSnappyCompression supported: 1 2024-06-30T13:04:32.653 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.merge_operator: 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_filter: None 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_filter_factory: None 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.table_factory: BlockBasedTable 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f0c67960d8) 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: cache_index_and_filter_blocks: 1 2024-06-30T13:04:32.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: pin_top_level_index_and_filter: 1 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: index_type: 0 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: data_block_index_type: 0 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: index_shortening: 1 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: data_block_hash_table_util_ratio: 0.750000 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: hash_index_allow_collision: 1 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: checksum: 1 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: no_block_cache: 0 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: block_cache: 0x55f0c67cef10 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: block_cache_name: BinnedLRUCache 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: block_cache_options: 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: capacity : 536870912 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: num_shard_bits : 4 2024-06-30T13:04:32.655 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: strict_capacity_limit : 0 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: high_pri_pool_ratio: 0.000 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: block_cache_compressed: (nil) 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: persistent_cache: (nil) 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: block_size: 4096 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: block_size_deviation: 10 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: block_restart_interval: 16 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: index_block_restart_interval: 1 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: metadata_block_size: 4096 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: partition_filters: 0 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: use_delta_encoding: 1 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: filter_policy: rocksdb.BuiltinBloomFilter 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: whole_key_filtering: 1 2024-06-30T13:04:32.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: verify_compression: 0 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: read_amp_bytes_per_bit: 0 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: format_version: 2 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: enable_index_compression: 1 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: block_align: 0 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.write_buffer_size: 33554432 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_write_buffer_number: 2 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compression: NoCompression 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.bottommost_compression: Disabled 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.prefix_extractor: nullptr 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.num_levels: 7 2024-06-30T13:04:32.657 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compression_opts.window_bits: -14 2024-06-30T13:04:32.658 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compression_opts.level: 32767 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compression_opts.strategy: 0 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compression_opts.enabled: false 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.target_file_size_base: 67108864 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.target_file_size_multiplier: 1 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-30T13:04:32.659 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.arena_block_size: 4194304 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-30T13:04:32.660 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.disable_auto_compactions: 0 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-30T13:04:32.661 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-30T13:04:32.662 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.table_properties_collectors: 2024-06-30T13:04:32.662 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.inplace_update_support: 0 2024-06-30T13:04:32.662 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-06-30T13:04:32.662 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-30T13:04:32.662 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-30T13:04:32.662 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.memtable_huge_page_size: 0 2024-06-30T13:04:32.662 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.bloom_locality: 0 2024-06-30T13:04:32.662 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.max_successive_merges: 0 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.paranoid_file_checks: 0 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.force_consistency_checks: 0 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.report_bg_io_stats: 0 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.ttl: 2592000 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.467+0000 7f0eeb545700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0eeb545700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1719752672474969, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0eeb545700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0eeb545700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-06-30T13:04:32.663 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0eeb545700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1719752672476123, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1437, "table_properties": {"data_size": 523, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1719752672, "oldest_key_time": 3, "file_creation_time": 0}} 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0eeb545700 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0eeb545700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1719752672476801, "job": 1, "event": "recovery_finished"} 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0eeb545700 4 rocksdb: DB pointer 0x55f0c6893800 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0ed235f700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0ed235f700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ** DB Stats ** 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Uptime(secs): 0.0 total, 0.0 interval 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-30T13:04:32.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ** Compaction Stats [default] ** 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ** Compaction Stats [default] ** 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Uptime(secs): 0.0 total, 0.0 interval 2024-06-30T13:04:32.665 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Flush(GB): cumulative 0.000, interval 0.000 2024-06-30T13:04:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-30T13:04:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: AddFile(Total Files): cumulative 0, interval 0 2024-06-30T13:04:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: AddFile(L0 Files): cumulative 0, interval 0 2024-06-30T13:04:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: AddFile(Keys): cumulative 0, interval 0 2024-06-30T13:04:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Cumulative compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:04:32.666 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Interval compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:04:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-30T13:04:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ** File Read Latency Histogram By Level [default] ** 2024-06-30T13:04:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ** Compaction Stats [default] ** 2024-06-30T13:04:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:04:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:04:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2024-06-30T13:04:32.667 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ** Compaction Stats [default] ** 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Uptime(secs): 0.0 total, 0.0 interval 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Flush(GB): cumulative 0.000, interval 0.000 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: AddFile(Total Files): cumulative 0, interval 0 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: AddFile(L0 Files): cumulative 0, interval 0 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: AddFile(Keys): cumulative 0, interval 0 2024-06-30T13:04:32.668 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Cumulative compaction: 0.00 GB write, 0.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: ** File Read Latency Histogram By Level [default] ** 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0eeb545700 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.471+0000 7f0eeb545700 0 using public_addr v2:172.21.15.167:0/0 -> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.475+0000 7f0eeb545700 0 starting mon.b rank -1 at public addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] at bind addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.475+0000 7f0eeb545700 1 mon.b@-1(???) e0 preinit fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 0 mon.b@-1(synchronizing).mds e1 new map 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 0 mon.b@-1(synchronizing).mds e1 print_map 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: e1 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: enable_multiple, ever_enabled_multiple: 1,1 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-30T13:04:32.669 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: legacy client fscid: -1 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: No filesystems configured 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 0 mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: cephadm 2024-06-30T13:04:30.817296+0000 mgr.y (mgr.14144) 19 : cephadm [INF] Deploying daemon mon.c on smithi144 2024-06-30T13:04:32.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.503+0000 7f0ed4b64700 1 mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta expand map: {default=false} 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta from 'false' to 'false' 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta expanded map: {default=false} 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta expand map: {default=info} 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta from 'info' to 'info' 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta expanded map: {default=info} 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta expand map: {default=daemon} 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta from 'daemon' to 'daemon' 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta expanded map: {default=daemon} 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta expand map: {default=debug} 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta from 'debug' to 'debug' 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 20 expand_channel_meta expanded map: {default=debug} 2024-06-30T13:04:32.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:32 smithi167 bash[14619]: debug 2024-06-30T13:04:32.507+0000 7f0ed4b64700 10 mon.b@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2024-06-30T13:04:32.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:32 smithi144 bash[14758]: cephadm 2024-06-30T13:04:30.817296+0000 mgr.y (mgr.14144) 19 : cephadm [INF] Deploying daemon mon.c on smithi144 2024-06-30T13:04:36.507 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:36 smithi144 bash[19242]: debug 2024-06-30T13:04:36.251+0000 7f3d4d1e1700 1 mon.c@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-06-30T13:04:37.828 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-30T13:04:37.828 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"f6ab1df6-36e0-11ef-bcaa-c7b262605968","modified":"2024-06-30T13:04:32.517490Z","created":"2024-06-30T13:02:52.974804Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-30T13:04:37.831 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-06-30T13:04:37.860 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:32.524075+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-30T13:04:37.860 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:32.524418+0000 mon.a (mon.0) 115 : cluster [INF] mon.a calling monitor election 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:32.527145+0000 mon.a (mon.0) 116 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:33.517251+0000 mon.a (mon.0) 117 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:34.306045+0000 mgr.y (mgr.14144) 21 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:34.516978+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:34.517161+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:35.517429+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:36.259805+0000 mon.a (mon.0) 120 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:36.306622+0000 mgr.y (mgr.14144) 22 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:36.517429+0000 mon.a (mon.0) 121 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:37.259806+0000 mon.a (mon.0) 122 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:37.517428+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:37.547065+0000 mon.a (mon.0) 124 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-30T13:04:37.861 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:37.552987+0000 mon.a (mon.0) 125 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],b=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} 2024-06-30T13:04:37.862 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:37.553100+0000 mon.a (mon.0) 126 : cluster [DBG] fsmap 2024-06-30T13:04:37.862 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:37.553160+0000 mon.a (mon.0) 127 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-06-30T13:04:37.862 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:37.553589+0000 mon.a (mon.0) 128 : cluster [DBG] mgrmap e13: y(active, since 41s) 2024-06-30T13:04:37.862 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: cluster 2024-06-30T13:04:37.560586+0000 mon.a (mon.0) 129 : cluster [INF] overall HEALTH_OK 2024-06-30T13:04:37.862 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:37.565350+0000 mon.a (mon.0) 130 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:37.862 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:37 smithi167 bash[14619]: audit 2024-06-30T13:04:37.566733+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:38.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:32.524075+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-30T13:04:38.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:32.524418+0000 mon.a (mon.0) 115 : cluster [INF] mon.a calling monitor election 2024-06-30T13:04:38.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:32.527145+0000 mon.a (mon.0) 116 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:38.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:33.517251+0000 mon.a (mon.0) 117 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:38.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:34.306045+0000 mgr.y (mgr.14144) 21 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:38.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:34.516978+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:38.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:34.517161+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-06-30T13:04:38.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:35.517429+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:36.259805+0000 mon.a (mon.0) 120 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:36.306622+0000 mgr.y (mgr.14144) 22 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:36.517429+0000 mon.a (mon.0) 121 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:37.259806+0000 mon.a (mon.0) 122 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:37.517428+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:37.547065+0000 mon.a (mon.0) 124 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:37.552987+0000 mon.a (mon.0) 125 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],b=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:37.553100+0000 mon.a (mon.0) 126 : cluster [DBG] fsmap 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:37.553160+0000 mon.a (mon.0) 127 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:37.553589+0000 mon.a (mon.0) 128 : cluster [DBG] mgrmap e13: y(active, since 41s) 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: cluster 2024-06-30T13:04:37.560586+0000 mon.a (mon.0) 129 : cluster [INF] overall HEALTH_OK 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:37.565350+0000 mon.a (mon.0) 130 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:38.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:37 smithi144 bash[14758]: audit 2024-06-30T13:04:37.566733+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:39.431 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-30T13:04:39.431 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph mon dump -f json 2024-06-30T13:04:43.593 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:38.267322+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-30T13:04:43.593 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:38.267934+0000 mon.a (mon.0) 135 : cluster [INF] mon.a calling monitor election 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:38.267957+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:38.270434+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:38.270815+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:38.307145+0000 mgr.y (mgr.14144) 23 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:39.260076+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:40.260500+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:40.263419+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:40.307666+0000 mgr.y (mgr.14144) 24 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:41.260571+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:42.260615+0000 mon.a (mon.0) 141 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:42.308086+0000 mgr.y (mgr.14144) 25 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:43.260774+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:43.291102+0000 mon.a (mon.0) 143 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:43.302149+0000 mon.a (mon.0) 144 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],b=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0],c=[v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0]} 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:43.302272+0000 mon.a (mon.0) 145 : cluster [DBG] fsmap 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:43.302348+0000 mon.a (mon.0) 146 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:43.302755+0000 mon.a (mon.0) 147 : cluster [DBG] mgrmap e13: y(active, since 47s) 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: cluster 2024-06-30T13:04:43.311665+0000 mon.a (mon.0) 148 : cluster [INF] overall HEALTH_OK 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:43.315703+0000 mon.a (mon.0) 149 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:43.319186+0000 mon.a (mon.0) 150 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:43.326410+0000 mon.a (mon.0) 151 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:43.328235+0000 mon.a (mon.0) 152 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:43.329581+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-30T13:04:43.595 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[14758]: audit 2024-06-30T13:04:43.330749+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:43.596 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:32.524075+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-30T13:04:43.596 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:32.524418+0000 mon.a (mon.0) 115 : cluster [INF] mon.a calling monitor election 2024-06-30T13:04:43.596 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:32.527145+0000 mon.a (mon.0) 116 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:43.596 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:33.517251+0000 mon.a (mon.0) 117 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:43.596 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:34.306045+0000 mgr.y (mgr.14144) 21 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.596 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:34.516978+0000 mon.a (mon.0) 118 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:43.596 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:34.517161+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-06-30T13:04:43.597 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:35.517429+0000 mon.a (mon.0) 119 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:43.597 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:36.259805+0000 mon.a (mon.0) 120 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.597 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:36.306622+0000 mgr.y (mgr.14144) 22 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.597 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:36.517429+0000 mon.a (mon.0) 121 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:43.597 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:37.259806+0000 mon.a (mon.0) 122 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.597 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:37.517428+0000 mon.a (mon.0) 123 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:43.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:37.547065+0000 mon.a (mon.0) 124 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-30T13:04:43.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:37.552987+0000 mon.a (mon.0) 125 : cluster [DBG] monmap e2: 2 mons at {a=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],b=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0]} 2024-06-30T13:04:43.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:37.553100+0000 mon.a (mon.0) 126 : cluster [DBG] fsmap 2024-06-30T13:04:43.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:37.553160+0000 mon.a (mon.0) 127 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-06-30T13:04:43.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:37.553589+0000 mon.a (mon.0) 128 : cluster [DBG] mgrmap e13: y(active, since 41s) 2024-06-30T13:04:43.598 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:37.560586+0000 mon.a (mon.0) 129 : cluster [INF] overall HEALTH_OK 2024-06-30T13:04:43.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:37.565350+0000 mon.a (mon.0) 130 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:37.566733+0000 mon.a (mon.0) 131 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:43.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:38.267322+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-30T13:04:43.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:38.267934+0000 mon.a (mon.0) 135 : cluster [INF] mon.a calling monitor election 2024-06-30T13:04:43.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:38.267957+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-06-30T13:04:43.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:38.270434+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:43.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:38.270815+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:38.307145+0000 mgr.y (mgr.14144) 23 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:39.260076+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:40.260500+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:40.263419+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-06-30T13:04:43.600 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:40.307666+0000 mgr.y (mgr.14144) 24 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:41.260571+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:42.260615+0000 mon.a (mon.0) 141 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:42.308086+0000 mgr.y (mgr.14144) 25 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:43.260774+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:43.291102+0000 mon.a (mon.0) 143 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-30T13:04:43.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:43.302149+0000 mon.a (mon.0) 144 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],b=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0],c=[v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0]} 2024-06-30T13:04:43.601 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:43.302272+0000 mon.a (mon.0) 145 : cluster [DBG] fsmap 2024-06-30T13:04:43.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:43.302348+0000 mon.a (mon.0) 146 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-06-30T13:04:43.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:43.302755+0000 mon.a (mon.0) 147 : cluster [DBG] mgrmap e13: y(active, since 47s) 2024-06-30T13:04:43.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: cluster 2024-06-30T13:04:43.311665+0000 mon.a (mon.0) 148 : cluster [INF] overall HEALTH_OK 2024-06-30T13:04:43.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:43.315703+0000 mon.a (mon.0) 149 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:43.319186+0000 mon.a (mon.0) 150 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:43.326410+0000 mon.a (mon.0) 151 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:43.328235+0000 mon.a (mon.0) 152 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:43.603 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:43.329581+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-30T13:04:43.603 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:43 smithi144 bash[19242]: audit 2024-06-30T13:04:43.330749+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:43.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:38.267322+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-30T13:04:43.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:38.267934+0000 mon.a (mon.0) 135 : cluster [INF] mon.a calling monitor election 2024-06-30T13:04:43.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:38.267957+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-06-30T13:04:43.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:38.270434+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-30T13:04:43.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:38.270815+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:38.307145+0000 mgr.y (mgr.14144) 23 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:39.260076+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:40.260500+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:40.263419+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:40.307666+0000 mgr.y (mgr.14144) 24 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:41.260571+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:42.260615+0000 mon.a (mon.0) 141 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:42.308086+0000 mgr.y (mgr.14144) 25 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:43.260774+0000 mon.a (mon.0) 142 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:43.291102+0000 mon.a (mon.0) 143 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:43.302149+0000 mon.a (mon.0) 144 : cluster [DBG] monmap e3: 3 mons at {a=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],b=[v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0],c=[v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0]} 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:43.302272+0000 mon.a (mon.0) 145 : cluster [DBG] fsmap 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:43.302348+0000 mon.a (mon.0) 146 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:43.302755+0000 mon.a (mon.0) 147 : cluster [DBG] mgrmap e13: y(active, since 47s) 2024-06-30T13:04:43.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: cluster 2024-06-30T13:04:43.311665+0000 mon.a (mon.0) 148 : cluster [INF] overall HEALTH_OK 2024-06-30T13:04:43.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:43.315703+0000 mon.a (mon.0) 149 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:43.319186+0000 mon.a (mon.0) 150 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:43.326410+0000 mon.a (mon.0) 151 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:43.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:43.328235+0000 mon.a (mon.0) 152 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:43.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:43.329581+0000 mon.a (mon.0) 153 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-30T13:04:43.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:43 smithi167 bash[14619]: audit 2024-06-30T13:04:43.330749+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:43.824 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-06-30T13:04:43.824 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"f6ab1df6-36e0-11ef-bcaa-c7b262605968","modified":"2024-06-30T13:04:38.261350Z","created":"2024-06-30T13:02:52.974804Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3301","nonce":0},{"type":"v1","addr":"172.21.15.144:6790","nonce":0}]},"addr":"172.21.15.144:6790/0","public_addr":"172.21.15.144:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-06-30T13:04:43.827 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-06-30T13:04:44.432 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-30T13:04:44.433 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph config generate-minimal-conf 2024-06-30T13:04:44.740 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:44 smithi144 bash[19242]: cephadm 2024-06-30T13:04:43.327809+0000 mgr.y (mgr.14144) 26 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-06-30T13:04:44.740 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:44 smithi144 bash[19242]: cephadm 2024-06-30T13:04:43.331794+0000 mgr.y (mgr.14144) 27 : cephadm [INF] Reconfiguring daemon mon.a on smithi144 2024-06-30T13:04:44.740 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:44 smithi144 bash[19242]: audit 2024-06-30T13:04:43.823410+0000 mon.a (mon.0) 155 : audit [DBG] from='client.? 172.21.15.167:0/3710889380' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-30T13:04:44.740 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:44 smithi144 bash[19242]: audit 2024-06-30T13:04:44.261235+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:44.740 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:44 smithi144 bash[14758]: cephadm 2024-06-30T13:04:43.327809+0000 mgr.y (mgr.14144) 26 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-06-30T13:04:44.741 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:44 smithi144 bash[14758]: cephadm 2024-06-30T13:04:43.331794+0000 mgr.y (mgr.14144) 27 : cephadm [INF] Reconfiguring daemon mon.a on smithi144 2024-06-30T13:04:44.741 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:44 smithi144 bash[14758]: audit 2024-06-30T13:04:43.823410+0000 mon.a (mon.0) 155 : audit [DBG] from='client.? 172.21.15.167:0/3710889380' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-30T13:04:44.741 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:44 smithi144 bash[14758]: audit 2024-06-30T13:04:44.261235+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:44.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:44 smithi167 bash[14619]: cephadm 2024-06-30T13:04:43.327809+0000 mgr.y (mgr.14144) 26 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-06-30T13:04:44.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:44 smithi167 bash[14619]: cephadm 2024-06-30T13:04:43.331794+0000 mgr.y (mgr.14144) 27 : cephadm [INF] Reconfiguring daemon mon.a on smithi144 2024-06-30T13:04:44.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:44 smithi167 bash[14619]: audit 2024-06-30T13:04:43.823410+0000 mon.a (mon.0) 155 : audit [DBG] from='client.? 172.21.15.167:0/3710889380' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-30T13:04:44.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:44 smithi167 bash[14619]: audit 2024-06-30T13:04:44.261235+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-30T13:04:45.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:45 smithi144 bash[14758]: cluster 2024-06-30T13:04:44.308514+0000 mgr.y (mgr.14144) 28 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:45.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:45 smithi144 bash[19242]: cluster 2024-06-30T13:04:44.308514+0000 mgr.y (mgr.14144) 28 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:45.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:45 smithi167 bash[14619]: cluster 2024-06-30T13:04:44.308514+0000 mgr.y (mgr.14144) 28 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:47.020 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:46 smithi144 bash[14758]: audit 2024-06-30T13:04:45.999853+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:47.021 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:46 smithi144 bash[14758]: cephadm 2024-06-30T13:04:46.000860+0000 mgr.y (mgr.14144) 29 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-06-30T13:04:47.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:46 smithi167 bash[14619]: audit 2024-06-30T13:04:45.999853+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:47.279 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:46 smithi167 bash[14619]: cephadm 2024-06-30T13:04:46.000860+0000 mgr.y (mgr.14144) 29 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-06-30T13:04:47.279 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:46 smithi167 bash[14619]: audit 2024-06-30T13:04:46.001429+0000 mon.a (mon.0) 158 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:47.279 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:46 smithi167 bash[14619]: audit 2024-06-30T13:04:46.002739+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-30T13:04:47.279 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:46 smithi167 bash[14619]: audit 2024-06-30T13:04:46.003897+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:47.279 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:46 smithi167 bash[14619]: cephadm 2024-06-30T13:04:46.005161+0000 mgr.y (mgr.14144) 30 : cephadm [INF] Reconfiguring daemon mon.c on smithi144 2024-06-30T13:04:47.279 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:46 smithi167 bash[14619]: cluster 2024-06-30T13:04:46.308863+0000 mgr.y (mgr.14144) 31 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:47.372 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[14758]: audit 2024-06-30T13:04:46.001429+0000 mon.a (mon.0) 158 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:47.372 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[14758]: audit 2024-06-30T13:04:46.002739+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-30T13:04:47.372 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[14758]: audit 2024-06-30T13:04:46.003897+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:47.372 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[14758]: cephadm 2024-06-30T13:04:46.005161+0000 mgr.y (mgr.14144) 30 : cephadm [INF] Reconfiguring daemon mon.c on smithi144 2024-06-30T13:04:47.372 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[14758]: cluster 2024-06-30T13:04:46.308863+0000 mgr.y (mgr.14144) 31 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:47.372 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[19242]: audit 2024-06-30T13:04:45.999853+0000 mon.a (mon.0) 157 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:47.372 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[19242]: cephadm 2024-06-30T13:04:46.000860+0000 mgr.y (mgr.14144) 29 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-06-30T13:04:47.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[19242]: audit 2024-06-30T13:04:46.001429+0000 mon.a (mon.0) 158 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:47.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[19242]: audit 2024-06-30T13:04:46.002739+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-30T13:04:47.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[19242]: audit 2024-06-30T13:04:46.003897+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:47.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[19242]: cephadm 2024-06-30T13:04:46.005161+0000 mgr.y (mgr.14144) 30 : cephadm [INF] Reconfiguring daemon mon.c on smithi144 2024-06-30T13:04:47.373 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:47 smithi144 bash[19242]: cluster 2024-06-30T13:04:46.308863+0000 mgr.y (mgr.14144) 31 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:49.729 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:49 smithi144 bash[14758]: cluster 2024-06-30T13:04:48.309332+0000 mgr.y (mgr.14144) 32 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:49.730 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:49 smithi144 bash[19242]: cluster 2024-06-30T13:04:48.309332+0000 mgr.y (mgr.14144) 32 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:49.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:49 smithi167 bash[14619]: cluster 2024-06-30T13:04:48.309332+0000 mgr.y (mgr.14144) 32 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:50.137 INFO:teuthology.orchestra.run.smithi144.stdout:# minimal ceph.conf for f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:04:50.138 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-06-30T13:04:50.138 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = f6ab1df6-36e0-11ef-bcaa-c7b262605968 2024-06-30T13:04:50.138 INFO:teuthology.orchestra.run.smithi144.stdout: mon_host = [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] [v2:172.21.15.144:3301/0,v1:172.21.15.144:6790/0] 2024-06-30T13:04:50.866 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-30T13:04:50.866 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T13:04:50.866 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-06-30T13:04:50.881 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T13:04:50.881 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-30T13:04:50.897 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-30T13:04:50.897 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-06-30T13:04:50.912 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-30T13:04:50.912 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-30T13:04:50.920 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:50 smithi167 bash[14619]: audit 2024-06-30T13:04:49.659273+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:50.920 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:50 smithi167 bash[14619]: cephadm 2024-06-30T13:04:49.660339+0000 mgr.y (mgr.14144) 33 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-06-30T13:04:50.920 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:50 smithi167 bash[14619]: audit 2024-06-30T13:04:49.660808+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:50.920 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:50 smithi167 bash[14619]: audit 2024-06-30T13:04:49.662363+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-30T13:04:50.920 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:50 smithi167 bash[14619]: audit 2024-06-30T13:04:49.664061+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:50.921 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:50 smithi167 bash[14619]: cephadm 2024-06-30T13:04:49.666111+0000 mgr.y (mgr.14144) 34 : cephadm [INF] Reconfiguring daemon mon.b on smithi167 2024-06-30T13:04:50.921 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:50 smithi167 bash[14619]: audit 2024-06-30T13:04:50.137295+0000 mon.a (mon.0) 165 : audit [DBG] from='client.? 172.21.15.144:0/3707582897' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:50.921 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:50 smithi167 bash[14619]: cluster 2024-06-30T13:04:50.309931+0000 mgr.y (mgr.14144) 35 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:50.926 INFO:tasks.cephadm:Adding mgr.y on smithi144 2024-06-30T13:04:50.926 INFO:tasks.cephadm:Adding mgr.x on smithi167 2024-06-30T13:04:50.927 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch apply mgr '2;smithi144=y;smithi167=x' 2024-06-30T13:04:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[14758]: audit 2024-06-30T13:04:49.659273+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[14758]: cephadm 2024-06-30T13:04:49.660339+0000 mgr.y (mgr.14144) 33 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-06-30T13:04:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[14758]: audit 2024-06-30T13:04:49.660808+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[14758]: audit 2024-06-30T13:04:49.662363+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-30T13:04:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[14758]: audit 2024-06-30T13:04:49.664061+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[14758]: cephadm 2024-06-30T13:04:49.666111+0000 mgr.y (mgr.14144) 34 : cephadm [INF] Reconfiguring daemon mon.b on smithi167 2024-06-30T13:04:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[14758]: audit 2024-06-30T13:04:50.137295+0000 mon.a (mon.0) 165 : audit [DBG] from='client.? 172.21.15.144:0/3707582897' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[14758]: cluster 2024-06-30T13:04:50.309931+0000 mgr.y (mgr.14144) 35 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:51.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[19242]: audit 2024-06-30T13:04:49.659273+0000 mon.a (mon.0) 161 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:51.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[19242]: cephadm 2024-06-30T13:04:49.660339+0000 mgr.y (mgr.14144) 33 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-06-30T13:04:51.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[19242]: audit 2024-06-30T13:04:49.660808+0000 mon.a (mon.0) 162 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-30T13:04:51.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[19242]: audit 2024-06-30T13:04:49.662363+0000 mon.a (mon.0) 163 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-30T13:04:51.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[19242]: audit 2024-06-30T13:04:49.664061+0000 mon.a (mon.0) 164 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:51.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[19242]: cephadm 2024-06-30T13:04:49.666111+0000 mgr.y (mgr.14144) 34 : cephadm [INF] Reconfiguring daemon mon.b on smithi167 2024-06-30T13:04:51.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[19242]: audit 2024-06-30T13:04:50.137295+0000 mon.a (mon.0) 165 : audit [DBG] from='client.? 172.21.15.144:0/3707582897' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:51.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:50 smithi144 bash[19242]: cluster 2024-06-30T13:04:50.309931+0000 mgr.y (mgr.14144) 35 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:53.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:52 smithi144 bash[14758]: audit 2024-06-30T13:04:51.721035+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:53.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:52 smithi144 bash[14758]: audit 2024-06-30T13:04:51.722647+0000 mon.a (mon.0) 167 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:53.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:52 smithi144 bash[14758]: audit 2024-06-30T13:04:51.730480+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:53.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:52 smithi144 bash[14758]: cluster 2024-06-30T13:04:52.310416+0000 mgr.y (mgr.14144) 36 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:53.008 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:52 smithi144 bash[19242]: audit 2024-06-30T13:04:51.721035+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:53.008 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:52 smithi144 bash[19242]: audit 2024-06-30T13:04:51.722647+0000 mon.a (mon.0) 167 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:53.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:52 smithi144 bash[19242]: audit 2024-06-30T13:04:51.730480+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:53.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:52 smithi144 bash[19242]: cluster 2024-06-30T13:04:52.310416+0000 mgr.y (mgr.14144) 36 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:52 smithi167 bash[14619]: audit 2024-06-30T13:04:51.721035+0000 mon.a (mon.0) 166 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:52 smithi167 bash[14619]: audit 2024-06-30T13:04:51.722647+0000 mon.a (mon.0) 167 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:52 smithi167 bash[14619]: audit 2024-06-30T13:04:51.730480+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:53.029 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:52 smithi167 bash[14619]: cluster 2024-06-30T13:04:52.310416+0000 mgr.y (mgr.14144) 36 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:54.238 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-06-30T13:04:55.057 DEBUG:teuthology.orchestra.run.smithi167:mgr.x> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.x.service 2024-06-30T13:04:55.060 INFO:tasks.cephadm:Deploying OSDs... 2024-06-30T13:04:55.060 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T13:04:55.060 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-06-30T13:04:55.068 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-30T13:04:55.068 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-06-30T13:04:55.075 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-30T13:04:55.075 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-30T13:04:55.075 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-06-30T13:04:55.075 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-30T13:04:55.075 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-06-30 13:00:35.674063089 +0000 2024-06-30T13:04:55.075 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-06-30 13:00:35.470067796 +0000 2024-06-30T13:04:55.075 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-06-30 13:00:35.470067796 +0000 2024-06-30T13:04:55.076 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-06-30T13:04:55.076 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-30T13:04:55.129 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-06-30T13:04:55.129 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-06-30T13:04:55.129 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000399871 s, 1.3 MB/s 2024-06-30T13:04:55.130 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-30T13:04:55.138 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-06-30T13:04:55.146 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-30T13:04:55.146 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-30T13:04:55.146 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 362 Links: 1 2024-06-30T13:04:55.146 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-30T13:04:55.146 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-06-30 13:00:36.106053125 +0000 2024-06-30T13:04:55.146 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-06-30 13:00:35.882058293 +0000 2024-06-30T13:04:55.146 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-06-30 13:00:35.882058293 +0000 2024-06-30T13:04:55.146 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-06-30T13:04:55.146 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-30T13:04:55.201 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-06-30T13:04:55.201 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-06-30T13:04:55.201 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000409047 s, 1.3 MB/s 2024-06-30T13:04:55.202 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-30T13:04:55.210 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-06-30T13:04:55.217 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-30T13:04:55.218 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-30T13:04:55.218 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 372 Links: 1 2024-06-30T13:04:55.218 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-30T13:04:55.218 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-06-30 13:00:36.362047220 +0000 2024-06-30T13:04:55.218 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-06-30 13:00:36.358047313 +0000 2024-06-30T13:04:55.218 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-06-30 13:00:36.358047313 +0000 2024-06-30T13:04:55.218 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-06-30T13:04:55.218 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-30T13:04:55.271 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-06-30T13:04:55.271 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-06-30T13:04:55.271 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000380924 s, 1.3 MB/s 2024-06-30T13:04:55.272 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-30T13:04:55.280 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-06-30T13:04:55.285 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-30T13:04:55.286 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-30T13:04:55.286 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-06-30T13:04:55.286 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-30T13:04:55.286 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-06-30 13:00:36.978033011 +0000 2024-06-30T13:04:55.286 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-06-30 13:00:36.774037716 +0000 2024-06-30T13:04:55.286 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-06-30 13:00:36.774037716 +0000 2024-06-30T13:04:55.286 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-06-30T13:04:55.286 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-30T13:04:55.343 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-06-30T13:04:55.343 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-06-30T13:04:55.343 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000369739 s, 1.4 MB/s 2024-06-30T13:04:55.344 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-30T13:04:55.351 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-30T13:04:55.351 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-06-30T13:04:55.358 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-30T13:04:55.359 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-06-30T13:04:55.407 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-30T13:04:55.408 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-30T13:04:55.408 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 359 Links: 1 2024-06-30T13:04:55.408 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-30T13:04:55.408 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-30 13:00:35.654569603 +0000 2024-06-30T13:04:55.408 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-30 13:00:35.442574491 +0000 2024-06-30T13:04:55.408 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-30 13:00:35.442574491 +0000 2024-06-30T13:04:55.408 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-06-30T13:04:55.408 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-30T13:04:55.419 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-30T13:04:55.419 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-30T13:04:55.419 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.00044984 s, 1.1 MB/s 2024-06-30T13:04:55.420 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-30T13:04:55.427 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-06-30T13:04:55.476 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-30T13:04:55.476 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-30T13:04:55.476 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 366 Links: 1 2024-06-30T13:04:55.476 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-30T13:04:55.476 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-30 13:00:36.054560384 +0000 2024-06-30T13:04:55.476 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-30 13:00:35.866564717 +0000 2024-06-30T13:04:55.476 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-30 13:00:35.866564717 +0000 2024-06-30T13:04:55.476 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-06-30T13:04:55.477 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-30T13:04:55.489 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: audit 2024-06-30T13:04:54.228433+0000 mgr.y (mgr.14144) 37 : audit [DBG] from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi144=y;smithi167=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: cephadm 2024-06-30T13:04:54.230073+0000 mgr.y (mgr.14144) 38 : cephadm [INF] Saving service mgr spec with placement smithi144=y;smithi167=x;count:2 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: audit 2024-06-30T13:04:54.236333+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: audit 2024-06-30T13:04:54.237802+0000 mon.a (mon.0) 170 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: audit 2024-06-30T13:04:54.246765+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: audit 2024-06-30T13:04:54.248476+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: audit 2024-06-30T13:04:54.252219+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: audit 2024-06-30T13:04:54.253688+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: audit 2024-06-30T13:04:54.255391+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: cephadm 2024-06-30T13:04:54.256642+0000 mgr.y (mgr.14144) 39 : cephadm [INF] Deploying daemon mgr.x on smithi167 2024-06-30T13:04:55.490 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:55 smithi167 bash[14619]: cluster 2024-06-30T13:04:54.310863+0000 mgr.y (mgr.14144) 40 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-30T13:04:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-30T13:04:55.492 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000432963 s, 1.2 MB/s 2024-06-30T13:04:55.493 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-30T13:04:55.499 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-06-30T13:04:55.505 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-30T13:04:55.505 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-30T13:04:55.505 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-06-30T13:04:55.505 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-30T13:04:55.505 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-30 13:00:36.482550519 +0000 2024-06-30T13:04:55.505 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-30 13:00:36.274555313 +0000 2024-06-30T13:04:55.505 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-30 13:00:36.274555313 +0000 2024-06-30T13:04:55.505 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-06-30T13:04:55.505 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-30T13:04:55.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: audit 2024-06-30T13:04:54.228433+0000 mgr.y (mgr.14144) 37 : audit [DBG] from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi144=y;smithi167=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:04:55.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: cephadm 2024-06-30T13:04:54.230073+0000 mgr.y (mgr.14144) 38 : cephadm [INF] Saving service mgr spec with placement smithi144=y;smithi167=x;count:2 2024-06-30T13:04:55.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: audit 2024-06-30T13:04:54.236333+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:55.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: audit 2024-06-30T13:04:54.237802+0000 mon.a (mon.0) 170 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:55.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: audit 2024-06-30T13:04:54.246765+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:55.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: audit 2024-06-30T13:04:54.248476+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-30T13:04:55.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: audit 2024-06-30T13:04:54.252219+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: audit 2024-06-30T13:04:54.253688+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: audit 2024-06-30T13:04:54.255391+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: cephadm 2024-06-30T13:04:54.256642+0000 mgr.y (mgr.14144) 39 : cephadm [INF] Deploying daemon mgr.x on smithi167 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[14758]: cluster 2024-06-30T13:04:54.310863+0000 mgr.y (mgr.14144) 40 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: audit 2024-06-30T13:04:54.228433+0000 mgr.y (mgr.14144) 37 : audit [DBG] from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi144=y;smithi167=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: cephadm 2024-06-30T13:04:54.230073+0000 mgr.y (mgr.14144) 38 : cephadm [INF] Saving service mgr spec with placement smithi144=y;smithi167=x;count:2 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: audit 2024-06-30T13:04:54.236333+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: audit 2024-06-30T13:04:54.237802+0000 mon.a (mon.0) 170 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: audit 2024-06-30T13:04:54.246765+0000 mon.a (mon.0) 171 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: audit 2024-06-30T13:04:54.248476+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: audit 2024-06-30T13:04:54.252219+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: audit 2024-06-30T13:04:54.253688+0000 mon.a (mon.0) 174 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-30T13:04:55.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: audit 2024-06-30T13:04:54.255391+0000 mon.a (mon.0) 175 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:04:55.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: cephadm 2024-06-30T13:04:54.256642+0000 mgr.y (mgr.14144) 39 : cephadm [INF] Deploying daemon mgr.x on smithi167 2024-06-30T13:04:55.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:55 smithi144 bash[19242]: cluster 2024-06-30T13:04:54.310863+0000 mgr.y (mgr.14144) 40 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:55.567 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-30T13:04:55.567 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-30T13:04:55.567 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000393598 s, 1.3 MB/s 2024-06-30T13:04:55.569 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-30T13:04:55.576 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-06-30T13:04:55.580 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-30T13:04:55.581 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-30T13:04:55.581 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-06-30T13:04:55.581 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-30T13:04:55.581 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-06-30 13:00:36.906540746 +0000 2024-06-30T13:04:55.581 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-06-30 13:00:36.714545170 +0000 2024-06-30T13:04:55.581 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-06-30 13:00:36.714545170 +0000 2024-06-30T13:04:55.581 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-06-30T13:04:55.581 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-30T13:04:55.639 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-06-30T13:04:55.640 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-06-30T13:04:55.640 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000367582 s, 1.4 MB/s 2024-06-30T13:04:55.641 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-30T13:04:55.649 INFO:tasks.cephadm:Deploying osd.0 on smithi144 with /dev/vg_nvme/lv_4... 2024-06-30T13:04:55.649 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-30T13:04:55.736 INFO:journalctl@ceph.mgr.x.smithi167.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:04:57.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[14758]: cluster 2024-06-30T13:04:56.311366+0000 mgr.y (mgr.14144) 41 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:57.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[14758]: audit 2024-06-30T13:04:56.498301+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:04:57.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[14758]: audit 2024-06-30T13:04:56.500589+0000 mon.a (mon.0) 177 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:04:57.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[14758]: audit 2024-06-30T13:04:56.967218+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:57.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[14758]: audit 2024-06-30T13:04:56.968690+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:57.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[19242]: cluster 2024-06-30T13:04:56.311366+0000 mgr.y (mgr.14144) 41 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:57.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[19242]: audit 2024-06-30T13:04:56.498301+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:04:57.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[19242]: audit 2024-06-30T13:04:56.500589+0000 mon.a (mon.0) 177 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:04:57.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[19242]: audit 2024-06-30T13:04:56.967218+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:57.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:57 smithi144 bash[19242]: audit 2024-06-30T13:04:56.968690+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:57 smithi167 bash[14619]: cluster 2024-06-30T13:04:56.311366+0000 mgr.y (mgr.14144) 41 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:57.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:57 smithi167 bash[14619]: audit 2024-06-30T13:04:56.498301+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:04:57.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:57 smithi167 bash[14619]: audit 2024-06-30T13:04:56.500589+0000 mon.a (mon.0) 177 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:04:57.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:57 smithi167 bash[14619]: audit 2024-06-30T13:04:56.967218+0000 mon.a (mon.0) 178 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:04:57.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:57 smithi167 bash[14619]: audit 2024-06-30T13:04:56.968690+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:04:58.278 INFO:journalctl@ceph.mgr.x.smithi167.stdout:Jun 30 13:04:58 smithi167 bash[15621]: debug 2024-06-30T13:04:58.098+0000 7f75d1a8d700 1 -- 172.21.15.167:0/2537310705 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558179d0a4e0 con 0x558179dfac00 2024-06-30T13:04:59.240 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-30T13:04:59.240 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-30T13:04:59.240 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-06-30T13:04:59.240 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-06-30T13:04:59.240 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0143864 s, 729 MB/s 2024-06-30T13:04:59.240 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-06-30T13:04:59.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:04:59 smithi144 bash[14758]: cluster 2024-06-30T13:04:58.311906+0000 mgr.y (mgr.14144) 42 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:59.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:04:59 smithi144 bash[19242]: cluster 2024-06-30T13:04:58.311906+0000 mgr.y (mgr.14144) 42 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:59.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:04:59 smithi167 bash[14619]: cluster 2024-06-30T13:04:58.311906+0000 mgr.y (mgr.14144) 42 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:04:59.930 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-06-30T13:05:01.472 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:01 smithi167 bash[14619]: audit 2024-06-30T13:05:00.030399+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:01.473 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:01 smithi167 bash[14619]: audit 2024-06-30T13:05:00.039492+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:01.473 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:01 smithi167 bash[14619]: cephadm 2024-06-30T13:05:00.041280+0000 mgr.y (mgr.14144) 43 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-06-30T13:05:01.473 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:01 smithi167 bash[14619]: audit 2024-06-30T13:05:00.041795+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-30T13:05:01.473 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:01 smithi167 bash[14619]: audit 2024-06-30T13:05:00.043289+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-30T13:05:01.473 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:01 smithi167 bash[14619]: audit 2024-06-30T13:05:00.044538+0000 mon.a (mon.0) 184 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:01.473 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:01 smithi167 bash[14619]: cephadm 2024-06-30T13:05:00.046007+0000 mgr.y (mgr.14144) 44 : cephadm [INF] Reconfiguring daemon mgr.y on smithi144 2024-06-30T13:05:01.473 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:01 smithi167 bash[14619]: cluster 2024-06-30T13:05:00.312356+0000 mgr.y (mgr.14144) 45 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:01.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[14758]: audit 2024-06-30T13:05:00.030399+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:01.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[14758]: audit 2024-06-30T13:05:00.039492+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:01.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[14758]: cephadm 2024-06-30T13:05:00.041280+0000 mgr.y (mgr.14144) 43 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-06-30T13:05:01.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[14758]: audit 2024-06-30T13:05:00.041795+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-30T13:05:01.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[14758]: audit 2024-06-30T13:05:00.043289+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-30T13:05:01.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[14758]: audit 2024-06-30T13:05:00.044538+0000 mon.a (mon.0) 184 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:01.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[14758]: cephadm 2024-06-30T13:05:00.046007+0000 mgr.y (mgr.14144) 44 : cephadm [INF] Reconfiguring daemon mgr.y on smithi144 2024-06-30T13:05:01.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[14758]: cluster 2024-06-30T13:05:00.312356+0000 mgr.y (mgr.14144) 45 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:01.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[19242]: audit 2024-06-30T13:05:00.030399+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:01.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[19242]: audit 2024-06-30T13:05:00.039492+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:01.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[19242]: cephadm 2024-06-30T13:05:00.041280+0000 mgr.y (mgr.14144) 43 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-06-30T13:05:01.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[19242]: audit 2024-06-30T13:05:00.041795+0000 mon.a (mon.0) 182 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-30T13:05:01.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[19242]: audit 2024-06-30T13:05:00.043289+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-30T13:05:01.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[19242]: audit 2024-06-30T13:05:00.044538+0000 mon.a (mon.0) 184 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:01.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[19242]: cephadm 2024-06-30T13:05:00.046007+0000 mgr.y (mgr.14144) 44 : cephadm [INF] Reconfiguring daemon mgr.y on smithi144 2024-06-30T13:05:01.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:01 smithi144 bash[19242]: cluster 2024-06-30T13:05:00.312356+0000 mgr.y (mgr.14144) 45 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:03.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:03 smithi144 bash[14758]: cluster 2024-06-30T13:05:02.312826+0000 mgr.y (mgr.14144) 46 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:03.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:03 smithi144 bash[14758]: audit 2024-06-30T13:05:03.347922+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:03.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:03 smithi144 bash[14758]: audit 2024-06-30T13:05:03.349424+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:03.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:03 smithi144 bash[14758]: audit 2024-06-30T13:05:03.356461+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:03.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:03 smithi144 bash[19242]: cluster 2024-06-30T13:05:02.312826+0000 mgr.y (mgr.14144) 46 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:03.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:03 smithi144 bash[19242]: audit 2024-06-30T13:05:03.347922+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:03.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:03 smithi144 bash[19242]: audit 2024-06-30T13:05:03.349424+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:03.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:03 smithi144 bash[19242]: audit 2024-06-30T13:05:03.356461+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:03.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:03 smithi167 bash[14619]: cluster 2024-06-30T13:05:02.312826+0000 mgr.y (mgr.14144) 46 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:03.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:03 smithi167 bash[14619]: audit 2024-06-30T13:05:03.347922+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:03.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:03 smithi167 bash[14619]: audit 2024-06-30T13:05:03.349424+0000 mon.a (mon.0) 186 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:03.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:03 smithi167 bash[14619]: audit 2024-06-30T13:05:03.356461+0000 mon.a (mon.0) 187 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:04.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[14758]: audit 2024-06-30T13:05:03.879547+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-30T13:05:04.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[14758]: cluster 2024-06-30T13:05:03.880100+0000 mon.a (mon.0) 188 : cluster [DBG] Standby manager daemon x started 2024-06-30T13:05:04.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[14758]: audit 2024-06-30T13:05:03.880643+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-30T13:05:04.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[14758]: audit 2024-06-30T13:05:03.882381+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-30T13:05:04.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[14758]: audit 2024-06-30T13:05:03.883100+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-30T13:05:04.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[19242]: audit 2024-06-30T13:05:03.879547+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-30T13:05:04.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[19242]: cluster 2024-06-30T13:05:03.880100+0000 mon.a (mon.0) 188 : cluster [DBG] Standby manager daemon x started 2024-06-30T13:05:04.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[19242]: audit 2024-06-30T13:05:03.880643+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-30T13:05:04.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[19242]: audit 2024-06-30T13:05:03.882381+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-30T13:05:04.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:04 smithi144 bash[19242]: audit 2024-06-30T13:05:03.883100+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-30T13:05:04.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:04 smithi167 bash[14619]: audit 2024-06-30T13:05:03.879547+0000 mon.b (mon.1) 3 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-30T13:05:04.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:04 smithi167 bash[14619]: cluster 2024-06-30T13:05:03.880100+0000 mon.a (mon.0) 188 : cluster [DBG] Standby manager daemon x started 2024-06-30T13:05:04.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:04 smithi167 bash[14619]: audit 2024-06-30T13:05:03.880643+0000 mon.b (mon.1) 4 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-30T13:05:04.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:04 smithi167 bash[14619]: audit 2024-06-30T13:05:03.882381+0000 mon.b (mon.1) 5 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-30T13:05:04.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:04 smithi167 bash[14619]: audit 2024-06-30T13:05:03.883100+0000 mon.b (mon.1) 6 : audit [DBG] from='mgr.? 172.21.15.167:0/1060032169' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-30T13:05:05.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[19242]: cluster 2024-06-30T13:05:04.313271+0000 mgr.y (mgr.14144) 47 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:05.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[19242]: cluster 2024-06-30T13:05:04.385006+0000 mon.a (mon.0) 189 : cluster [DBG] mgrmap e14: y(active, since 68s), standbys: x 2024-06-30T13:05:05.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[19242]: audit 2024-06-30T13:05:04.385199+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-30T13:05:05.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[19242]: audit 2024-06-30T13:05:04.730096+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:05:05.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[19242]: audit 2024-06-30T13:05:04.733255+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:05:05.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[19242]: audit 2024-06-30T13:05:04.734705+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:05.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[14758]: cluster 2024-06-30T13:05:04.313271+0000 mgr.y (mgr.14144) 47 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[14758]: cluster 2024-06-30T13:05:04.385006+0000 mon.a (mon.0) 189 : cluster [DBG] mgrmap e14: y(active, since 68s), standbys: x 2024-06-30T13:05:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[14758]: audit 2024-06-30T13:05:04.385199+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-30T13:05:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[14758]: audit 2024-06-30T13:05:04.730096+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:05:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[14758]: audit 2024-06-30T13:05:04.733255+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:05:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:05 smithi144 bash[14758]: audit 2024-06-30T13:05:04.734705+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:05.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:05 smithi167 bash[14619]: cluster 2024-06-30T13:05:04.313271+0000 mgr.y (mgr.14144) 47 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:05.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:05 smithi167 bash[14619]: cluster 2024-06-30T13:05:04.385006+0000 mon.a (mon.0) 189 : cluster [DBG] mgrmap e14: y(active, since 68s), standbys: x 2024-06-30T13:05:05.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:05 smithi167 bash[14619]: audit 2024-06-30T13:05:04.385199+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-30T13:05:05.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:05 smithi167 bash[14619]: audit 2024-06-30T13:05:04.730096+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:05:05.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:05 smithi167 bash[14619]: audit 2024-06-30T13:05:04.733255+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:05:05.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:05 smithi167 bash[14619]: audit 2024-06-30T13:05:04.734705+0000 mon.a (mon.0) 193 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:06.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:06 smithi144 bash[14758]: audit 2024-06-30T13:05:04.728507+0000 mgr.y (mgr.14144) 48 : audit [DBG] from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:05:06.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:06 smithi144 bash[14758]: cephadm 2024-06-30T13:05:04.732810+0000 mgr.y (mgr.14144) 49 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:05:06.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:06 smithi144 bash[19242]: audit 2024-06-30T13:05:04.728507+0000 mgr.y (mgr.14144) 48 : audit [DBG] from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:05:06.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:06 smithi144 bash[19242]: cephadm 2024-06-30T13:05:04.732810+0000 mgr.y (mgr.14144) 49 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:05:06.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:06 smithi167 bash[14619]: audit 2024-06-30T13:05:04.728507+0000 mgr.y (mgr.14144) 48 : audit [DBG] from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:05:06.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:06 smithi167 bash[14619]: cephadm 2024-06-30T13:05:04.732810+0000 mgr.y (mgr.14144) 49 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:05:07.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:07 smithi144 bash[14758]: cluster 2024-06-30T13:05:06.313668+0000 mgr.y (mgr.14144) 50 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:07.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:07 smithi144 bash[19242]: cluster 2024-06-30T13:05:06.313668+0000 mgr.y (mgr.14144) 50 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:07.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:07 smithi167 bash[14619]: cluster 2024-06-30T13:05:06.313668+0000 mgr.y (mgr.14144) 50 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[14758]: cluster 2024-06-30T13:05:08.314084+0000 mgr.y (mgr.14144) 51 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[14758]: audit 2024-06-30T13:05:08.925376+0000 mon.a (mon.0) 194 : audit [INF] from='client.? 172.21.15.144:0/2312002932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cabe81-91cc-44b6-a500-526db10d84ba"}]: dispatch 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[14758]: audit 2024-06-30T13:05:08.932437+0000 mon.a (mon.0) 195 : audit [INF] from='client.? 172.21.15.144:0/2312002932' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86cabe81-91cc-44b6-a500-526db10d84ba"}]': finished 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[14758]: cluster 2024-06-30T13:05:08.932590+0000 mon.a (mon.0) 196 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[14758]: audit 2024-06-30T13:05:08.932750+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[19242]: cluster 2024-06-30T13:05:08.314084+0000 mgr.y (mgr.14144) 51 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[19242]: audit 2024-06-30T13:05:08.925376+0000 mon.a (mon.0) 194 : audit [INF] from='client.? 172.21.15.144:0/2312002932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cabe81-91cc-44b6-a500-526db10d84ba"}]: dispatch 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[19242]: audit 2024-06-30T13:05:08.932437+0000 mon.a (mon.0) 195 : audit [INF] from='client.? 172.21.15.144:0/2312002932' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86cabe81-91cc-44b6-a500-526db10d84ba"}]': finished 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[19242]: cluster 2024-06-30T13:05:08.932590+0000 mon.a (mon.0) 196 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-06-30T13:05:09.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:09 smithi144 bash[19242]: audit 2024-06-30T13:05:08.932750+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:09.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:09 smithi167 bash[14619]: cluster 2024-06-30T13:05:08.314084+0000 mgr.y (mgr.14144) 51 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:09.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:09 smithi167 bash[14619]: audit 2024-06-30T13:05:08.925376+0000 mon.a (mon.0) 194 : audit [INF] from='client.? 172.21.15.144:0/2312002932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "86cabe81-91cc-44b6-a500-526db10d84ba"}]: dispatch 2024-06-30T13:05:09.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:09 smithi167 bash[14619]: audit 2024-06-30T13:05:08.932437+0000 mon.a (mon.0) 195 : audit [INF] from='client.? 172.21.15.144:0/2312002932' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "86cabe81-91cc-44b6-a500-526db10d84ba"}]': finished 2024-06-30T13:05:09.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:09 smithi167 bash[14619]: cluster 2024-06-30T13:05:08.932590+0000 mon.a (mon.0) 196 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-06-30T13:05:09.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:09 smithi167 bash[14619]: audit 2024-06-30T13:05:08.932750+0000 mon.a (mon.0) 197 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:10.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:10 smithi144 bash[14758]: audit 2024-06-30T13:05:09.595497+0000 mon.a (mon.0) 198 : audit [DBG] from='client.? 172.21.15.144:0/1095765663' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:05:10.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:10 smithi144 bash[19242]: audit 2024-06-30T13:05:09.595497+0000 mon.a (mon.0) 198 : audit [DBG] from='client.? 172.21.15.144:0/1095765663' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:05:10.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:10 smithi167 bash[14619]: audit 2024-06-30T13:05:09.595497+0000 mon.a (mon.0) 198 : audit [DBG] from='client.? 172.21.15.144:0/1095765663' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:05:11.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:11 smithi144 bash[14758]: cluster 2024-06-30T13:05:10.314531+0000 mgr.y (mgr.14144) 52 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:11.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:11 smithi144 bash[19242]: cluster 2024-06-30T13:05:10.314531+0000 mgr.y (mgr.14144) 52 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:11.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:11 smithi167 bash[14619]: cluster 2024-06-30T13:05:10.314531+0000 mgr.y (mgr.14144) 52 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:12.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:12 smithi144 bash[14758]: cluster 2024-06-30T13:05:12.314910+0000 mgr.y (mgr.14144) 53 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:12.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:12 smithi144 bash[19242]: cluster 2024-06-30T13:05:12.314910+0000 mgr.y (mgr.14144) 53 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:12.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:12 smithi167 bash[14619]: cluster 2024-06-30T13:05:12.314910+0000 mgr.y (mgr.14144) 53 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:15.692 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:15 smithi144 bash[19242]: cluster 2024-06-30T13:05:14.315406+0000 mgr.y (mgr.14144) 54 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:15.692 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:15 smithi144 bash[14758]: cluster 2024-06-30T13:05:14.315406+0000 mgr.y (mgr.14144) 54 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:15 smithi167 bash[14619]: cluster 2024-06-30T13:05:14.315406+0000 mgr.y (mgr.14144) 54 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:17.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:17 smithi144 bash[14758]: cluster 2024-06-30T13:05:16.315874+0000 mgr.y (mgr.14144) 55 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:17.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:17 smithi144 bash[14758]: audit 2024-06-30T13:05:16.846603+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-30T13:05:17.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:17 smithi144 bash[14758]: audit 2024-06-30T13:05:16.848218+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:17.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:17 smithi144 bash[19242]: cluster 2024-06-30T13:05:16.315874+0000 mgr.y (mgr.14144) 55 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:17.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:17 smithi144 bash[19242]: audit 2024-06-30T13:05:16.846603+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-30T13:05:17.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:17 smithi144 bash[19242]: audit 2024-06-30T13:05:16.848218+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:17.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:17 smithi167 bash[14619]: cluster 2024-06-30T13:05:16.315874+0000 mgr.y (mgr.14144) 55 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:17.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:17 smithi167 bash[14619]: audit 2024-06-30T13:05:16.846603+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-30T13:05:17.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:17 smithi167 bash[14619]: audit 2024-06-30T13:05:16.848218+0000 mon.a (mon.0) 200 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:18.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:18 smithi144 bash[14758]: cephadm 2024-06-30T13:05:16.849653+0000 mgr.y (mgr.14144) 56 : cephadm [INF] Deploying daemon osd.0 on smithi144 2024-06-30T13:05:18.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:18 smithi144 bash[19242]: cephadm 2024-06-30T13:05:16.849653+0000 mgr.y (mgr.14144) 56 : cephadm [INF] Deploying daemon osd.0 on smithi144 2024-06-30T13:05:18.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:18 smithi167 bash[14619]: cephadm 2024-06-30T13:05:16.849653+0000 mgr.y (mgr.14144) 56 : cephadm [INF] Deploying daemon osd.0 on smithi144 2024-06-30T13:05:19.544 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 0 on host 'smithi144' 2024-06-30T13:05:19.648 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:19 smithi144 bash[14758]: cluster 2024-06-30T13:05:18.316358+0000 mgr.y (mgr.14144) 57 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:19.648 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:19 smithi144 bash[19242]: cluster 2024-06-30T13:05:18.316358+0000 mgr.y (mgr.14144) 57 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:19.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:19 smithi167 bash[14619]: cluster 2024-06-30T13:05:18.316358+0000 mgr.y (mgr.14144) 57 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:20.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:20 smithi144 bash[14758]: audit 2024-06-30T13:05:19.528432+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:20.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:20 smithi144 bash[14758]: audit 2024-06-30T13:05:19.533042+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:20.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:20 smithi144 bash[14758]: audit 2024-06-30T13:05:19.541439+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:20.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:20 smithi144 bash[19242]: audit 2024-06-30T13:05:19.528432+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:20.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:20 smithi144 bash[19242]: audit 2024-06-30T13:05:19.533042+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:20.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:20 smithi144 bash[19242]: audit 2024-06-30T13:05:19.541439+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:20.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:20 smithi167 bash[14619]: audit 2024-06-30T13:05:19.528432+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:20.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:20 smithi167 bash[14619]: audit 2024-06-30T13:05:19.533042+0000 mon.a (mon.0) 202 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:20.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:20 smithi167 bash[14619]: audit 2024-06-30T13:05:19.541439+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:20.804 DEBUG:teuthology.orchestra.run.smithi144:osd.0> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.0.service 2024-06-30T13:05:20.807 INFO:tasks.cephadm:Deploying osd.1 on smithi144 with /dev/vg_nvme/lv_3... 2024-06-30T13:05:20.807 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-30T13:05:21.191 INFO:journalctl@ceph.osd.0.smithi144.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:05:21.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:21 smithi144 bash[14758]: cluster 2024-06-30T13:05:20.316855+0000 mgr.y (mgr.14144) 58 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:21.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:21 smithi144 bash[19242]: cluster 2024-06-30T13:05:20.316855+0000 mgr.y (mgr.14144) 58 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:21.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:21 smithi167 bash[14619]: cluster 2024-06-30T13:05:20.316855+0000 mgr.y (mgr.14144) 58 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:22.660 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:22 smithi144 bash[21647]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-30T13:05:22.660 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:22 smithi144 bash[21647]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2024-06-30T13:05:22.660 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:22 smithi144 bash[21647]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0/block 2024-06-30T13:05:22.660 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:22 smithi144 bash[21647]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block 2024-06-30T13:05:22.660 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:22 smithi144 bash[21647]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-30T13:05:22.660 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:22 smithi144 bash[21647]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-06-30T13:05:22.660 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:22 smithi144 bash[21647]: --> ceph-volume lvm activate successful for osd ID: 0 2024-06-30T13:05:23.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:23 smithi144 bash[19242]: cluster 2024-06-30T13:05:22.317295+0000 mgr.y (mgr.14144) 59 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:23.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:23 smithi144 bash[19242]: audit 2024-06-30T13:05:22.764229+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:23.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:23 smithi144 bash[14758]: cluster 2024-06-30T13:05:22.317295+0000 mgr.y (mgr.14144) 59 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:23.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:23 smithi144 bash[14758]: audit 2024-06-30T13:05:22.764229+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:23.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:23 smithi167 bash[14619]: cluster 2024-06-30T13:05:22.317295+0000 mgr.y (mgr.14144) 59 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:23.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:23 smithi167 bash[14619]: audit 2024-06-30T13:05:22.764229+0000 mon.a (mon.0) 204 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:25.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:25 smithi144 bash[14758]: cluster 2024-06-30T13:05:24.317784+0000 mgr.y (mgr.14144) 60 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:25.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:25 smithi144 bash[19242]: cluster 2024-06-30T13:05:24.317784+0000 mgr.y (mgr.14144) 60 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:25.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:25 smithi167 bash[14619]: cluster 2024-06-30T13:05:24.317784+0000 mgr.y (mgr.14144) 60 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:25.928 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:25 smithi144 bash[21982]: debug 2024-06-30T13:05:25.634+0000 7f6ff3fb7700 1 -- 172.21.15.144:0/1210781586 <== mon.2 v2:172.21.15.144:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5569ace2e4e0 con 0x5569acea6c00 2024-06-30T13:05:26.007 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:25 smithi144 bash[21982]: debug 2024-06-30T13:05:25.922+0000 7f6ffba42080 -1 Falling back to public interface 2024-06-30T13:05:26.738 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-30T13:05:26.738 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-30T13:05:26.738 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-06-30T13:05:26.738 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-06-30T13:05:26.738 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0194935 s, 538 MB/s 2024-06-30T13:05:26.738 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-06-30T13:05:27.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:27 smithi144 bash[19242]: cluster 2024-06-30T13:05:26.318248+0000 mgr.y (mgr.14144) 61 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:27.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:27 smithi144 bash[14758]: cluster 2024-06-30T13:05:26.318248+0000 mgr.y (mgr.14144) 61 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:27.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:27 smithi167 bash[14619]: cluster 2024-06-30T13:05:26.318248+0000 mgr.y (mgr.14144) 61 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:27.821 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-06-30T13:05:28.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:28 smithi144 bash[19242]: cluster 2024-06-30T13:05:28.318709+0000 mgr.y (mgr.14144) 62 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:28.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:28 smithi144 bash[14758]: cluster 2024-06-30T13:05:28.318709+0000 mgr.y (mgr.14144) 62 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:28.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:28 smithi167 bash[14619]: cluster 2024-06-30T13:05:28.318709+0000 mgr.y (mgr.14144) 62 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:29.257 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:28 smithi144 bash[21982]: debug 2024-06-30T13:05:28.966+0000 7f6ffba42080 -1 osd.0 0 log_to_monitors {default=true} 2024-06-30T13:05:29.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:29 smithi144 bash[19242]: audit 2024-06-30T13:05:28.974769+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 [v2:172.21.15.144:6802/2331490331,v1:172.21.15.144:6803/2331490331]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-30T13:05:29.599 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:29 smithi144 bash[19242]: audit 2024-06-30T13:05:28.975259+0000 mon.a (mon.0) 205 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-30T13:05:29.600 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:29 smithi144 bash[14758]: audit 2024-06-30T13:05:28.974769+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 [v2:172.21.15.144:6802/2331490331,v1:172.21.15.144:6803/2331490331]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-30T13:05:29.600 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:29 smithi144 bash[14758]: audit 2024-06-30T13:05:28.975259+0000 mon.a (mon.0) 205 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-30T13:05:29.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:29 smithi167 bash[14619]: audit 2024-06-30T13:05:28.974769+0000 mon.c (mon.2) 2 : audit [INF] from='osd.0 [v2:172.21.15.144:6802/2331490331,v1:172.21.15.144:6803/2331490331]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-30T13:05:29.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:29 smithi167 bash[14619]: audit 2024-06-30T13:05:28.975259+0000 mon.a (mon.0) 205 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-30T13:05:30.532 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[14758]: audit 2024-06-30T13:05:29.432420+0000 mon.a (mon.0) 206 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-30T13:05:30.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:30 smithi167 bash[14619]: audit 2024-06-30T13:05:29.432420+0000 mon.a (mon.0) 206 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-30T13:05:30.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:30 smithi167 bash[14619]: cluster 2024-06-30T13:05:29.432497+0000 mon.a (mon.0) 207 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-06-30T13:05:30.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:30 smithi167 bash[14619]: audit 2024-06-30T13:05:29.432670+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:30.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:30 smithi167 bash[14619]: audit 2024-06-30T13:05:29.435129+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 [v2:172.21.15.144:6802/2331490331,v1:172.21.15.144:6803/2331490331]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:05:30.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:30 smithi167 bash[14619]: audit 2024-06-30T13:05:29.435716+0000 mon.a (mon.0) 209 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:05:30.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:30 smithi167 bash[14619]: cluster 2024-06-30T13:05:30.319164+0000 mgr.y (mgr.14144) 63 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:31.007 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[21982]: debug 2024-06-30T13:05:30.442+0000 7f6ff1fb3700 -1 osd.0 0 waiting for initial osdmap 2024-06-30T13:05:31.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[14758]: cluster 2024-06-30T13:05:29.432497+0000 mon.a (mon.0) 207 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-06-30T13:05:31.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[14758]: audit 2024-06-30T13:05:29.432670+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:31.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[14758]: audit 2024-06-30T13:05:29.435129+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 [v2:172.21.15.144:6802/2331490331,v1:172.21.15.144:6803/2331490331]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:05:31.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[14758]: audit 2024-06-30T13:05:29.435716+0000 mon.a (mon.0) 209 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:05:31.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[14758]: cluster 2024-06-30T13:05:30.319164+0000 mgr.y (mgr.14144) 63 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:31.008 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[19242]: audit 2024-06-30T13:05:29.432420+0000 mon.a (mon.0) 206 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-30T13:05:31.008 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[19242]: cluster 2024-06-30T13:05:29.432497+0000 mon.a (mon.0) 207 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-06-30T13:05:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[19242]: audit 2024-06-30T13:05:29.432670+0000 mon.a (mon.0) 208 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[19242]: audit 2024-06-30T13:05:29.435129+0000 mon.c (mon.2) 3 : audit [INF] from='osd.0 [v2:172.21.15.144:6802/2331490331,v1:172.21.15.144:6803/2331490331]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:05:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[19242]: audit 2024-06-30T13:05:29.435716+0000 mon.a (mon.0) 209 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:05:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:30 smithi144 bash[19242]: cluster 2024-06-30T13:05:30.319164+0000 mgr.y (mgr.14144) 63 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:31.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[19242]: audit 2024-06-30T13:05:30.437104+0000 mon.a (mon.0) 210 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:05:31.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[19242]: cluster 2024-06-30T13:05:30.437433+0000 mon.a (mon.0) 211 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-06-30T13:05:31.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[19242]: audit 2024-06-30T13:05:30.438132+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:31.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[19242]: audit 2024-06-30T13:05:30.446128+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:31.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[19242]: cephadm 2024-06-30T13:05:31.397589+0000 mgr.y (mgr.14144) 64 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:05:31.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[19242]: audit 2024-06-30T13:05:31.406521+0000 mon.a (mon.0) 214 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:31.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[14758]: audit 2024-06-30T13:05:30.437104+0000 mon.a (mon.0) 210 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:05:31.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[14758]: cluster 2024-06-30T13:05:30.437433+0000 mon.a (mon.0) 211 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-06-30T13:05:31.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[14758]: audit 2024-06-30T13:05:30.438132+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:31.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[14758]: audit 2024-06-30T13:05:30.446128+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:31.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[14758]: cephadm 2024-06-30T13:05:31.397589+0000 mgr.y (mgr.14144) 64 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:05:31.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:31 smithi144 bash[14758]: audit 2024-06-30T13:05:31.406521+0000 mon.a (mon.0) 214 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:31.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:31 smithi167 bash[14619]: audit 2024-06-30T13:05:30.437104+0000 mon.a (mon.0) 210 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:05:31.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:31 smithi167 bash[14619]: cluster 2024-06-30T13:05:30.437433+0000 mon.a (mon.0) 211 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-06-30T13:05:31.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:31 smithi167 bash[14619]: audit 2024-06-30T13:05:30.438132+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:31.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:31 smithi167 bash[14619]: audit 2024-06-30T13:05:30.446128+0000 mon.a (mon.0) 213 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:31.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:31 smithi167 bash[14619]: cephadm 2024-06-30T13:05:31.397589+0000 mgr.y (mgr.14144) 64 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:05:31.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:31 smithi167 bash[14619]: audit 2024-06-30T13:05:31.406521+0000 mon.a (mon.0) 214 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:32.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: cluster 2024-06-30T13:05:29.924008+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:05:32.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: cluster 2024-06-30T13:05:29.924214+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:05:32.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: cluster 2024-06-30T13:05:31.446816+0000 mon.a (mon.0) 215 : cluster [INF] osd.0 [v2:172.21.15.144:6802/2331490331,v1:172.21.15.144:6803/2331490331] boot 2024-06-30T13:05:32.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: cluster 2024-06-30T13:05:31.446968+0000 mon.a (mon.0) 216 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-06-30T13:05:32.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: audit 2024-06-30T13:05:31.447837+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:32.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: audit 2024-06-30T13:05:31.678014+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: audit 2024-06-30T13:05:31.680465+0000 mon.a (mon.0) 219 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: audit 2024-06-30T13:05:31.686747+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: audit 2024-06-30T13:05:32.009365+0000 mgr.y (mgr.14144) 65 : audit [DBG] from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: audit 2024-06-30T13:05:32.010413+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: cephadm 2024-06-30T13:05:32.012470+0000 mgr.y (mgr.14144) 66 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: audit 2024-06-30T13:05:32.012903+0000 mon.a (mon.0) 222 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: audit 2024-06-30T13:05:32.014198+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[14758]: cluster 2024-06-30T13:05:32.319648+0000 mgr.y (mgr.14144) 67 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: cluster 2024-06-30T13:05:29.924008+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: cluster 2024-06-30T13:05:29.924214+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: cluster 2024-06-30T13:05:31.446816+0000 mon.a (mon.0) 215 : cluster [INF] osd.0 [v2:172.21.15.144:6802/2331490331,v1:172.21.15.144:6803/2331490331] boot 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: cluster 2024-06-30T13:05:31.446968+0000 mon.a (mon.0) 216 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-06-30T13:05:32.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: audit 2024-06-30T13:05:31.447837+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:32.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: audit 2024-06-30T13:05:31.678014+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:32.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: audit 2024-06-30T13:05:31.680465+0000 mon.a (mon.0) 219 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:32.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: audit 2024-06-30T13:05:31.686747+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:32.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: audit 2024-06-30T13:05:32.009365+0000 mgr.y (mgr.14144) 65 : audit [DBG] from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:05:32.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: audit 2024-06-30T13:05:32.010413+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:05:32.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: cephadm 2024-06-30T13:05:32.012470+0000 mgr.y (mgr.14144) 66 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:05:32.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: audit 2024-06-30T13:05:32.012903+0000 mon.a (mon.0) 222 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:05:32.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: audit 2024-06-30T13:05:32.014198+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:32.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:32 smithi144 bash[19242]: cluster 2024-06-30T13:05:32.319648+0000 mgr.y (mgr.14144) 67 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: cluster 2024-06-30T13:05:29.924008+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: cluster 2024-06-30T13:05:29.924214+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: cluster 2024-06-30T13:05:31.446816+0000 mon.a (mon.0) 215 : cluster [INF] osd.0 [v2:172.21.15.144:6802/2331490331,v1:172.21.15.144:6803/2331490331] boot 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: cluster 2024-06-30T13:05:31.446968+0000 mon.a (mon.0) 216 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: audit 2024-06-30T13:05:31.447837+0000 mon.a (mon.0) 217 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: audit 2024-06-30T13:05:31.678014+0000 mon.a (mon.0) 218 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: audit 2024-06-30T13:05:31.680465+0000 mon.a (mon.0) 219 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: audit 2024-06-30T13:05:31.686747+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: audit 2024-06-30T13:05:32.009365+0000 mgr.y (mgr.14144) 65 : audit [DBG] from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: audit 2024-06-30T13:05:32.010413+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:05:32.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: cephadm 2024-06-30T13:05:32.012470+0000 mgr.y (mgr.14144) 66 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:05:32.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: audit 2024-06-30T13:05:32.012903+0000 mon.a (mon.0) 222 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:05:32.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: audit 2024-06-30T13:05:32.014198+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:32.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:32 smithi167 bash[14619]: cluster 2024-06-30T13:05:32.319648+0000 mgr.y (mgr.14144) 67 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-30T13:05:34.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:33 smithi144 bash[14758]: cluster 2024-06-30T13:05:32.691402+0000 mon.a (mon.0) 224 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-06-30T13:05:34.008 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:33 smithi144 bash[19242]: cluster 2024-06-30T13:05:32.691402+0000 mon.a (mon.0) 224 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-06-30T13:05:34.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:33 smithi167 bash[14619]: cluster 2024-06-30T13:05:32.691402+0000 mon.a (mon.0) 224 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-06-30T13:05:35.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:35 smithi144 bash[19242]: cluster 2024-06-30T13:05:34.320175+0000 mgr.y (mgr.14144) 68 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:35.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:35 smithi144 bash[14758]: cluster 2024-06-30T13:05:34.320175+0000 mgr.y (mgr.14144) 68 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:35.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:35 smithi167 bash[14619]: cluster 2024-06-30T13:05:34.320175+0000 mgr.y (mgr.14144) 68 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:36.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:36 smithi144 bash[14758]: audit 2024-06-30T13:05:36.120233+0000 mon.a (mon.0) 225 : audit [INF] from='client.? 172.21.15.144:0/1245351255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e13d329a-1644-44d9-9f7c-0bbe011e620c"}]: dispatch 2024-06-30T13:05:36.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:36 smithi144 bash[14758]: audit 2024-06-30T13:05:36.129637+0000 mon.a (mon.0) 226 : audit [INF] from='client.? 172.21.15.144:0/1245351255' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e13d329a-1644-44d9-9f7c-0bbe011e620c"}]': finished 2024-06-30T13:05:36.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:36 smithi144 bash[14758]: cluster 2024-06-30T13:05:36.129772+0000 mon.a (mon.0) 227 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-06-30T13:05:36.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:36 smithi144 bash[14758]: audit 2024-06-30T13:05:36.130005+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:36.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:36 smithi144 bash[19242]: audit 2024-06-30T13:05:36.120233+0000 mon.a (mon.0) 225 : audit [INF] from='client.? 172.21.15.144:0/1245351255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e13d329a-1644-44d9-9f7c-0bbe011e620c"}]: dispatch 2024-06-30T13:05:36.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:36 smithi144 bash[19242]: audit 2024-06-30T13:05:36.129637+0000 mon.a (mon.0) 226 : audit [INF] from='client.? 172.21.15.144:0/1245351255' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e13d329a-1644-44d9-9f7c-0bbe011e620c"}]': finished 2024-06-30T13:05:36.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:36 smithi144 bash[19242]: cluster 2024-06-30T13:05:36.129772+0000 mon.a (mon.0) 227 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-06-30T13:05:36.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:36 smithi144 bash[19242]: audit 2024-06-30T13:05:36.130005+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:36.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:36 smithi167 bash[14619]: audit 2024-06-30T13:05:36.120233+0000 mon.a (mon.0) 225 : audit [INF] from='client.? 172.21.15.144:0/1245351255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e13d329a-1644-44d9-9f7c-0bbe011e620c"}]: dispatch 2024-06-30T13:05:36.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:36 smithi167 bash[14619]: audit 2024-06-30T13:05:36.129637+0000 mon.a (mon.0) 226 : audit [INF] from='client.? 172.21.15.144:0/1245351255' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e13d329a-1644-44d9-9f7c-0bbe011e620c"}]': finished 2024-06-30T13:05:36.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:36 smithi167 bash[14619]: cluster 2024-06-30T13:05:36.129772+0000 mon.a (mon.0) 227 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-06-30T13:05:36.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:36 smithi167 bash[14619]: audit 2024-06-30T13:05:36.130005+0000 mon.a (mon.0) 228 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:37.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:37 smithi144 bash[14758]: cluster 2024-06-30T13:05:36.320660+0000 mgr.y (mgr.14144) 69 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:37.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:37 smithi144 bash[14758]: audit 2024-06-30T13:05:36.762984+0000 mon.a (mon.0) 229 : audit [DBG] from='client.? 172.21.15.144:0/614006484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:05:37.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:37 smithi144 bash[19242]: cluster 2024-06-30T13:05:36.320660+0000 mgr.y (mgr.14144) 69 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:37.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:37 smithi144 bash[19242]: audit 2024-06-30T13:05:36.762984+0000 mon.a (mon.0) 229 : audit [DBG] from='client.? 172.21.15.144:0/614006484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:05:37.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:37 smithi167 bash[14619]: cluster 2024-06-30T13:05:36.320660+0000 mgr.y (mgr.14144) 69 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:37.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:37 smithi167 bash[14619]: audit 2024-06-30T13:05:36.762984+0000 mon.a (mon.0) 229 : audit [DBG] from='client.? 172.21.15.144:0/614006484' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:05:39.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:39 smithi144 bash[14758]: cluster 2024-06-30T13:05:38.321206+0000 mgr.y (mgr.14144) 70 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:39.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:39 smithi144 bash[19242]: cluster 2024-06-30T13:05:38.321206+0000 mgr.y (mgr.14144) 70 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:39.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:39 smithi167 bash[14619]: cluster 2024-06-30T13:05:38.321206+0000 mgr.y (mgr.14144) 70 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:41.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:41 smithi144 bash[14758]: cluster 2024-06-30T13:05:40.321753+0000 mgr.y (mgr.14144) 71 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:41.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:41 smithi144 bash[19242]: cluster 2024-06-30T13:05:40.321753+0000 mgr.y (mgr.14144) 71 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:41.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:41 smithi167 bash[14619]: cluster 2024-06-30T13:05:40.321753+0000 mgr.y (mgr.14144) 71 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:43.698 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:43 smithi144 bash[14758]: cluster 2024-06-30T13:05:42.322225+0000 mgr.y (mgr.14144) 72 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:43.698 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:43 smithi144 bash[19242]: cluster 2024-06-30T13:05:42.322225+0000 mgr.y (mgr.14144) 72 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:43.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:43 smithi167 bash[14619]: cluster 2024-06-30T13:05:42.322225+0000 mgr.y (mgr.14144) 72 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:44.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:44 smithi144 bash[14758]: audit 2024-06-30T13:05:44.207879+0000 mon.a (mon.0) 230 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-30T13:05:44.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:44 smithi144 bash[14758]: audit 2024-06-30T13:05:44.209458+0000 mon.a (mon.0) 231 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:44.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:44 smithi144 bash[19242]: audit 2024-06-30T13:05:44.207879+0000 mon.a (mon.0) 230 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-30T13:05:44.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:44 smithi144 bash[19242]: audit 2024-06-30T13:05:44.209458+0000 mon.a (mon.0) 231 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:44.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:44 smithi167 bash[14619]: audit 2024-06-30T13:05:44.207879+0000 mon.a (mon.0) 230 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-30T13:05:44.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:44 smithi167 bash[14619]: audit 2024-06-30T13:05:44.209458+0000 mon.a (mon.0) 231 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:05:45.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:45 smithi144 bash[19242]: cephadm 2024-06-30T13:05:44.210896+0000 mgr.y (mgr.14144) 73 : cephadm [INF] Deploying daemon osd.1 on smithi144 2024-06-30T13:05:45.658 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:45 smithi144 bash[19242]: cluster 2024-06-30T13:05:44.322671+0000 mgr.y (mgr.14144) 74 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:45.659 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:45 smithi144 bash[14758]: cephadm 2024-06-30T13:05:44.210896+0000 mgr.y (mgr.14144) 73 : cephadm [INF] Deploying daemon osd.1 on smithi144 2024-06-30T13:05:45.659 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:45 smithi144 bash[14758]: cluster 2024-06-30T13:05:44.322671+0000 mgr.y (mgr.14144) 74 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:45.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:45 smithi167 bash[14619]: cephadm 2024-06-30T13:05:44.210896+0000 mgr.y (mgr.14144) 73 : cephadm [INF] Deploying daemon osd.1 on smithi144 2024-06-30T13:05:45.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:45 smithi167 bash[14619]: cluster 2024-06-30T13:05:44.322671+0000 mgr.y (mgr.14144) 74 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:46.823 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 1 on host 'smithi144' 2024-06-30T13:05:47.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:47 smithi144 bash[14758]: cluster 2024-06-30T13:05:46.323132+0000 mgr.y (mgr.14144) 75 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:47.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:47 smithi144 bash[14758]: audit 2024-06-30T13:05:46.808838+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:47.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:47 smithi144 bash[14758]: audit 2024-06-30T13:05:46.813481+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:47.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:47 smithi144 bash[14758]: audit 2024-06-30T13:05:46.822154+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:47 smithi144 bash[19242]: cluster 2024-06-30T13:05:46.323132+0000 mgr.y (mgr.14144) 75 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:47 smithi144 bash[19242]: audit 2024-06-30T13:05:46.808838+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:47 smithi144 bash[19242]: audit 2024-06-30T13:05:46.813481+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:47 smithi144 bash[19242]: audit 2024-06-30T13:05:46.822154+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:47.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:47 smithi167 bash[14619]: cluster 2024-06-30T13:05:46.323132+0000 mgr.y (mgr.14144) 75 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:47.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:47 smithi167 bash[14619]: audit 2024-06-30T13:05:46.808838+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:47.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:47 smithi167 bash[14619]: audit 2024-06-30T13:05:46.813481+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:47.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:47 smithi167 bash[14619]: audit 2024-06-30T13:05:46.822154+0000 mon.a (mon.0) 234 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:48.103 DEBUG:teuthology.orchestra.run.smithi144:osd.1> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.1.service 2024-06-30T13:05:48.105 INFO:tasks.cephadm:Deploying osd.2 on smithi144 with /dev/vg_nvme/lv_2... 2024-06-30T13:05:48.105 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-30T13:05:48.554 INFO:journalctl@ceph.osd.1.smithi144.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:05:49.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:49 smithi144 bash[14758]: cluster 2024-06-30T13:05:48.323637+0000 mgr.y (mgr.14144) 76 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:49.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:49 smithi144 bash[19242]: cluster 2024-06-30T13:05:48.323637+0000 mgr.y (mgr.14144) 76 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:49.758 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:49 smithi144 bash[24341]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-30T13:05:49.758 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:49 smithi144 bash[24341]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2024-06-30T13:05:49.758 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:49 smithi144 bash[24341]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-1/block 2024-06-30T13:05:49.758 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:49 smithi144 bash[24341]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-1/block 2024-06-30T13:05:49.758 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:49 smithi144 bash[24341]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-30T13:05:49.759 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:49 smithi144 bash[24341]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-06-30T13:05:49.759 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:49 smithi144 bash[24341]: --> ceph-volume lvm activate successful for osd ID: 1 2024-06-30T13:05:49.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:49 smithi167 bash[14619]: cluster 2024-06-30T13:05:48.323637+0000 mgr.y (mgr.14144) 76 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:51.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:50 smithi144 bash[14758]: audit 2024-06-30T13:05:49.957766+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:51.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:50 smithi144 bash[14758]: cluster 2024-06-30T13:05:50.324220+0000 mgr.y (mgr.14144) 77 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:51.258 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:50 smithi144 bash[19242]: audit 2024-06-30T13:05:49.957766+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:51.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:50 smithi144 bash[19242]: cluster 2024-06-30T13:05:50.324220+0000 mgr.y (mgr.14144) 77 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:51.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:50 smithi167 bash[14619]: audit 2024-06-30T13:05:49.957766+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:51.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:50 smithi167 bash[14619]: cluster 2024-06-30T13:05:50.324220+0000 mgr.y (mgr.14144) 77 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:53.596 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:53 smithi144 bash[14758]: cluster 2024-06-30T13:05:52.324712+0000 mgr.y (mgr.14144) 78 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:53.596 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:53 smithi144 bash[19242]: cluster 2024-06-30T13:05:52.324712+0000 mgr.y (mgr.14144) 78 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:53.597 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:53 smithi144 bash[24683]: debug 2024-06-30T13:05:53.290+0000 7f9d1f77e700 1 -- 172.21.15.144:0/283873397 <== mon.0 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b67ba824e0 con 0x55b67bafac00 2024-06-30T13:05:53.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:53 smithi167 bash[14619]: cluster 2024-06-30T13:05:52.324712+0000 mgr.y (mgr.14144) 78 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:54.008 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:53 smithi144 bash[24683]: debug 2024-06-30T13:05:53.590+0000 7f9d27209080 -1 Falling back to public interface 2024-06-30T13:05:54.933 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-30T13:05:54.934 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-30T13:05:54.934 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-06-30T13:05:54.934 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-06-30T13:05:54.934 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0153714 s, 682 MB/s 2024-06-30T13:05:54.934 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-06-30T13:05:55.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:55 smithi144 bash[14758]: cluster 2024-06-30T13:05:54.325393+0000 mgr.y (mgr.14144) 79 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:55.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:55 smithi144 bash[19242]: cluster 2024-06-30T13:05:54.325393+0000 mgr.y (mgr.14144) 79 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:55 smithi167 bash[14619]: cluster 2024-06-30T13:05:54.325393+0000 mgr.y (mgr.14144) 79 : cluster [DBG] pgmap v59: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:56.007 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-06-30T13:05:57.008 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:56 smithi144 bash[24683]: debug 2024-06-30T13:05:56.634+0000 7f9d27209080 -1 osd.1 0 log_to_monitors {default=true} 2024-06-30T13:05:57.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:57 smithi144 bash[19242]: cluster 2024-06-30T13:05:56.325862+0000 mgr.y (mgr.14144) 80 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:57.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:57 smithi144 bash[19242]: audit 2024-06-30T13:05:56.500373+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:05:57.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:57 smithi144 bash[19242]: audit 2024-06-30T13:05:56.502366+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:05:57.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:57 smithi144 bash[19242]: audit 2024-06-30T13:05:56.643416+0000 mon.a (mon.0) 238 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-30T13:05:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:57 smithi144 bash[14758]: cluster 2024-06-30T13:05:56.325862+0000 mgr.y (mgr.14144) 80 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:57.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:57 smithi144 bash[14758]: audit 2024-06-30T13:05:56.500373+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:05:57.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:57 smithi144 bash[14758]: audit 2024-06-30T13:05:56.502366+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:05:57.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:57 smithi144 bash[14758]: audit 2024-06-30T13:05:56.643416+0000 mon.a (mon.0) 238 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-30T13:05:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:57 smithi167 bash[14619]: cluster 2024-06-30T13:05:56.325862+0000 mgr.y (mgr.14144) 80 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:57 smithi167 bash[14619]: audit 2024-06-30T13:05:56.500373+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:05:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:57 smithi167 bash[14619]: audit 2024-06-30T13:05:56.502366+0000 mon.a (mon.0) 237 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:05:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:57 smithi167 bash[14619]: audit 2024-06-30T13:05:56.643416+0000 mon.a (mon.0) 238 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-30T13:05:58.758 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[24683]: debug 2024-06-30T13:05:58.402+0000 7f9d1d77a700 -1 osd.1 0 waiting for initial osdmap 2024-06-30T13:05:58.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[14758]: audit 2024-06-30T13:05:57.396659+0000 mon.a (mon.0) 239 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-30T13:05:58.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[14758]: cluster 2024-06-30T13:05:57.396822+0000 mon.a (mon.0) 240 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-06-30T13:05:58.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[14758]: audit 2024-06-30T13:05:57.397851+0000 mon.a (mon.0) 241 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:58.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[14758]: audit 2024-06-30T13:05:57.399321+0000 mon.a (mon.0) 242 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[14758]: audit 2024-06-30T13:05:58.398087+0000 mon.a (mon.0) 243 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[14758]: cluster 2024-06-30T13:05:58.398450+0000 mon.a (mon.0) 244 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[14758]: audit 2024-06-30T13:05:58.398726+0000 mon.a (mon.0) 245 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[19242]: audit 2024-06-30T13:05:57.396659+0000 mon.a (mon.0) 239 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[19242]: cluster 2024-06-30T13:05:57.396822+0000 mon.a (mon.0) 240 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[19242]: audit 2024-06-30T13:05:57.397851+0000 mon.a (mon.0) 241 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[19242]: audit 2024-06-30T13:05:57.399321+0000 mon.a (mon.0) 242 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[19242]: audit 2024-06-30T13:05:58.398087+0000 mon.a (mon.0) 243 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[19242]: cluster 2024-06-30T13:05:58.398450+0000 mon.a (mon.0) 244 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-06-30T13:05:58.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:58 smithi144 bash[19242]: audit 2024-06-30T13:05:58.398726+0000 mon.a (mon.0) 245 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:58.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:58 smithi167 bash[14619]: audit 2024-06-30T13:05:57.396659+0000 mon.a (mon.0) 239 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-30T13:05:58.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:58 smithi167 bash[14619]: cluster 2024-06-30T13:05:57.396822+0000 mon.a (mon.0) 240 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-06-30T13:05:58.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:58 smithi167 bash[14619]: audit 2024-06-30T13:05:57.397851+0000 mon.a (mon.0) 241 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:58.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:58 smithi167 bash[14619]: audit 2024-06-30T13:05:57.399321+0000 mon.a (mon.0) 242 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:05:58.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:58 smithi167 bash[14619]: audit 2024-06-30T13:05:58.398087+0000 mon.a (mon.0) 243 : audit [INF] from='osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:05:58.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:58 smithi167 bash[14619]: cluster 2024-06-30T13:05:58.398450+0000 mon.a (mon.0) 244 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-06-30T13:05:58.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:58 smithi167 bash[14619]: audit 2024-06-30T13:05:58.398726+0000 mon.a (mon.0) 245 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:59.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[14758]: cluster 2024-06-30T13:05:58.326386+0000 mgr.y (mgr.14144) 81 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:59.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[14758]: audit 2024-06-30T13:05:58.409465+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:59.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[14758]: audit 2024-06-30T13:05:59.217015+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:59.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[14758]: audit 2024-06-30T13:05:59.223244+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:59.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[14758]: audit 2024-06-30T13:05:59.225320+0000 mon.a (mon.0) 249 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:59.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[14758]: audit 2024-06-30T13:05:59.231999+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:59.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[19242]: cluster 2024-06-30T13:05:58.326386+0000 mgr.y (mgr.14144) 81 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:59.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[19242]: audit 2024-06-30T13:05:58.409465+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:59.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[19242]: audit 2024-06-30T13:05:59.217015+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:59.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[19242]: audit 2024-06-30T13:05:59.223244+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:59.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[19242]: audit 2024-06-30T13:05:59.225320+0000 mon.a (mon.0) 249 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:59.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:05:59 smithi144 bash[19242]: audit 2024-06-30T13:05:59.231999+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:59.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:59 smithi167 bash[14619]: cluster 2024-06-30T13:05:58.326386+0000 mgr.y (mgr.14144) 81 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:05:59.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:59 smithi167 bash[14619]: audit 2024-06-30T13:05:58.409465+0000 mon.a (mon.0) 246 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:05:59.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:59 smithi167 bash[14619]: audit 2024-06-30T13:05:59.217015+0000 mon.a (mon.0) 247 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:59.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:59 smithi167 bash[14619]: audit 2024-06-30T13:05:59.223244+0000 mon.a (mon.0) 248 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:05:59.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:59 smithi167 bash[14619]: audit 2024-06-30T13:05:59.225320+0000 mon.a (mon.0) 249 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:05:59.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:05:59 smithi167 bash[14619]: audit 2024-06-30T13:05:59.231999+0000 mon.a (mon.0) 250 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:00.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: cluster 2024-06-30T13:05:57.631337+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:06:00.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: cluster 2024-06-30T13:05:57.631603+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:06:00.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: cephadm 2024-06-30T13:05:59.210566+0000 mgr.y (mgr.14144) 82 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:06:00.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: audit 2024-06-30T13:05:59.412155+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: cluster 2024-06-30T13:05:59.418702+0000 mon.a (mon.0) 252 : cluster [INF] osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033] boot 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: cluster 2024-06-30T13:05:59.418773+0000 mon.a (mon.0) 253 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: audit 2024-06-30T13:05:59.419302+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: audit 2024-06-30T13:06:00.323350+0000 mon.a (mon.0) 255 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: audit 2024-06-30T13:06:00.326401+0000 mon.a (mon.0) 256 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[19242]: audit 2024-06-30T13:06:00.328497+0000 mon.a (mon.0) 257 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: cluster 2024-06-30T13:05:57.631337+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: cluster 2024-06-30T13:05:57.631603+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: cephadm 2024-06-30T13:05:59.210566+0000 mgr.y (mgr.14144) 82 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: audit 2024-06-30T13:05:59.412155+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: cluster 2024-06-30T13:05:59.418702+0000 mon.a (mon.0) 252 : cluster [INF] osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033] boot 2024-06-30T13:06:00.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: cluster 2024-06-30T13:05:59.418773+0000 mon.a (mon.0) 253 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-06-30T13:06:00.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: audit 2024-06-30T13:05:59.419302+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:06:00.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: audit 2024-06-30T13:06:00.323350+0000 mon.a (mon.0) 255 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:06:00.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: audit 2024-06-30T13:06:00.326401+0000 mon.a (mon.0) 256 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:06:00.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:00 smithi144 bash[14758]: audit 2024-06-30T13:06:00.328497+0000 mon.a (mon.0) 257 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:00.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: cluster 2024-06-30T13:05:57.631337+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:06:00.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: cluster 2024-06-30T13:05:57.631603+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:06:00.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: cephadm 2024-06-30T13:05:59.210566+0000 mgr.y (mgr.14144) 82 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:06:00.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: audit 2024-06-30T13:05:59.412155+0000 mon.a (mon.0) 251 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:06:00.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: cluster 2024-06-30T13:05:59.418702+0000 mon.a (mon.0) 252 : cluster [INF] osd.1 [v2:172.21.15.144:6810/2344135033,v1:172.21.15.144:6811/2344135033] boot 2024-06-30T13:06:00.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: cluster 2024-06-30T13:05:59.418773+0000 mon.a (mon.0) 253 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-06-30T13:06:00.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: audit 2024-06-30T13:05:59.419302+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-30T13:06:00.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: audit 2024-06-30T13:06:00.323350+0000 mon.a (mon.0) 255 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:06:00.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: audit 2024-06-30T13:06:00.326401+0000 mon.a (mon.0) 256 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:06:00.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:00 smithi167 bash[14619]: audit 2024-06-30T13:06:00.328497+0000 mon.a (mon.0) 257 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:01.712 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:01 smithi144 bash[14758]: audit 2024-06-30T13:06:00.321743+0000 mgr.y (mgr.14144) 83 : audit [DBG] from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:06:01.712 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:01 smithi144 bash[14758]: cephadm 2024-06-30T13:06:00.325912+0000 mgr.y (mgr.14144) 84 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:06:01.712 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:01 smithi144 bash[14758]: cluster 2024-06-30T13:06:00.326796+0000 mgr.y (mgr.14144) 85 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:06:01.712 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:01 smithi144 bash[14758]: cluster 2024-06-30T13:06:00.426222+0000 mon.a (mon.0) 258 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-06-30T13:06:01.713 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:01 smithi144 bash[19242]: audit 2024-06-30T13:06:00.321743+0000 mgr.y (mgr.14144) 83 : audit [DBG] from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:06:01.713 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:01 smithi144 bash[19242]: cephadm 2024-06-30T13:06:00.325912+0000 mgr.y (mgr.14144) 84 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:06:01.713 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:01 smithi144 bash[19242]: cluster 2024-06-30T13:06:00.326796+0000 mgr.y (mgr.14144) 85 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:06:01.713 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:01 smithi144 bash[19242]: cluster 2024-06-30T13:06:00.426222+0000 mon.a (mon.0) 258 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-06-30T13:06:01.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:01 smithi167 bash[14619]: audit 2024-06-30T13:06:00.321743+0000 mgr.y (mgr.14144) 83 : audit [DBG] from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:06:01.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:01 smithi167 bash[14619]: cephadm 2024-06-30T13:06:00.325912+0000 mgr.y (mgr.14144) 84 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:06:01.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:01 smithi167 bash[14619]: cluster 2024-06-30T13:06:00.326796+0000 mgr.y (mgr.14144) 85 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2024-06-30T13:06:01.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:01 smithi167 bash[14619]: cluster 2024-06-30T13:06:00.426222+0000 mon.a (mon.0) 258 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-06-30T13:06:03.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:03 smithi144 bash[14758]: cluster 2024-06-30T13:06:02.327287+0000 mgr.y (mgr.14144) 86 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 9.5 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:03.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:03 smithi144 bash[19242]: cluster 2024-06-30T13:06:02.327287+0000 mgr.y (mgr.14144) 86 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 9.5 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:03.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:03 smithi167 bash[14619]: cluster 2024-06-30T13:06:02.327287+0000 mgr.y (mgr.14144) 86 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 9.5 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:05.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[14758]: cluster 2024-06-30T13:06:04.327642+0000 mgr.y (mgr.14144) 87 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:05.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[14758]: audit 2024-06-30T13:06:04.382133+0000 mon.c (mon.2) 4 : audit [INF] from='client.? 172.21.15.144:0/709221966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9aac72-52c3-4c6a-92ae-cd0ad5033be5"}]: dispatch 2024-06-30T13:06:05.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[14758]: audit 2024-06-30T13:06:04.382781+0000 mon.a (mon.0) 259 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9aac72-52c3-4c6a-92ae-cd0ad5033be5"}]: dispatch 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[14758]: audit 2024-06-30T13:06:04.395815+0000 mon.a (mon.0) 260 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed9aac72-52c3-4c6a-92ae-cd0ad5033be5"}]': finished 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[14758]: cluster 2024-06-30T13:06:04.395919+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[14758]: audit 2024-06-30T13:06:04.396165+0000 mon.a (mon.0) 262 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[14758]: audit 2024-06-30T13:06:05.067375+0000 mon.a (mon.0) 263 : audit [DBG] from='client.? 172.21.15.144:0/3362190888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[19242]: cluster 2024-06-30T13:06:04.327642+0000 mgr.y (mgr.14144) 87 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[19242]: audit 2024-06-30T13:06:04.382133+0000 mon.c (mon.2) 4 : audit [INF] from='client.? 172.21.15.144:0/709221966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9aac72-52c3-4c6a-92ae-cd0ad5033be5"}]: dispatch 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[19242]: audit 2024-06-30T13:06:04.382781+0000 mon.a (mon.0) 259 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9aac72-52c3-4c6a-92ae-cd0ad5033be5"}]: dispatch 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[19242]: audit 2024-06-30T13:06:04.395815+0000 mon.a (mon.0) 260 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed9aac72-52c3-4c6a-92ae-cd0ad5033be5"}]': finished 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[19242]: cluster 2024-06-30T13:06:04.395919+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-06-30T13:06:05.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[19242]: audit 2024-06-30T13:06:04.396165+0000 mon.a (mon.0) 262 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:05.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:05 smithi144 bash[19242]: audit 2024-06-30T13:06:05.067375+0000 mon.a (mon.0) 263 : audit [DBG] from='client.? 172.21.15.144:0/3362190888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:06:05.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:05 smithi167 bash[14619]: cluster 2024-06-30T13:06:04.327642+0000 mgr.y (mgr.14144) 87 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:05.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:05 smithi167 bash[14619]: audit 2024-06-30T13:06:04.382133+0000 mon.c (mon.2) 4 : audit [INF] from='client.? 172.21.15.144:0/709221966' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9aac72-52c3-4c6a-92ae-cd0ad5033be5"}]: dispatch 2024-06-30T13:06:05.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:05 smithi167 bash[14619]: audit 2024-06-30T13:06:04.382781+0000 mon.a (mon.0) 259 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ed9aac72-52c3-4c6a-92ae-cd0ad5033be5"}]: dispatch 2024-06-30T13:06:05.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:05 smithi167 bash[14619]: audit 2024-06-30T13:06:04.395815+0000 mon.a (mon.0) 260 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ed9aac72-52c3-4c6a-92ae-cd0ad5033be5"}]': finished 2024-06-30T13:06:05.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:05 smithi167 bash[14619]: cluster 2024-06-30T13:06:04.395919+0000 mon.a (mon.0) 261 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-06-30T13:06:05.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:05 smithi167 bash[14619]: audit 2024-06-30T13:06:04.396165+0000 mon.a (mon.0) 262 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:05.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:05 smithi167 bash[14619]: audit 2024-06-30T13:06:05.067375+0000 mon.a (mon.0) 263 : audit [DBG] from='client.? 172.21.15.144:0/3362190888' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:06:07.726 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:07 smithi144 bash[19242]: cluster 2024-06-30T13:06:06.328132+0000 mgr.y (mgr.14144) 88 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:07.726 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:07 smithi144 bash[14758]: cluster 2024-06-30T13:06:06.328132+0000 mgr.y (mgr.14144) 88 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:07.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:07 smithi167 bash[14619]: cluster 2024-06-30T13:06:06.328132+0000 mgr.y (mgr.14144) 88 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:09.624 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:09 smithi144 bash[14758]: cluster 2024-06-30T13:06:08.328625+0000 mgr.y (mgr.14144) 89 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:09.624 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:09 smithi144 bash[19242]: cluster 2024-06-30T13:06:08.328625+0000 mgr.y (mgr.14144) 89 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:09.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:09 smithi167 bash[14619]: cluster 2024-06-30T13:06:08.328625+0000 mgr.y (mgr.14144) 89 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:11.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:11 smithi144 bash[14758]: cluster 2024-06-30T13:06:10.329202+0000 mgr.y (mgr.14144) 90 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:11.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:11 smithi144 bash[19242]: cluster 2024-06-30T13:06:10.329202+0000 mgr.y (mgr.14144) 90 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:11.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:11 smithi167 bash[14619]: cluster 2024-06-30T13:06:10.329202+0000 mgr.y (mgr.14144) 90 : cluster [DBG] pgmap v72: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:13.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:13 smithi144 bash[14758]: cluster 2024-06-30T13:06:12.329706+0000 mgr.y (mgr.14144) 91 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:13.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:13 smithi144 bash[14758]: audit 2024-06-30T13:06:12.722489+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-30T13:06:13.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:13 smithi144 bash[14758]: audit 2024-06-30T13:06:12.724100+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:13.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:13 smithi144 bash[19242]: cluster 2024-06-30T13:06:12.329706+0000 mgr.y (mgr.14144) 91 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:13.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:13 smithi144 bash[19242]: audit 2024-06-30T13:06:12.722489+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-30T13:06:13.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:13 smithi144 bash[19242]: audit 2024-06-30T13:06:12.724100+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:13 smithi167 bash[14619]: cluster 2024-06-30T13:06:12.329706+0000 mgr.y (mgr.14144) 91 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:13 smithi167 bash[14619]: audit 2024-06-30T13:06:12.722489+0000 mon.a (mon.0) 264 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-30T13:06:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:13 smithi167 bash[14619]: audit 2024-06-30T13:06:12.724100+0000 mon.a (mon.0) 265 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:14.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:14 smithi144 bash[14758]: cephadm 2024-06-30T13:06:12.725444+0000 mgr.y (mgr.14144) 92 : cephadm [INF] Deploying daemon osd.2 on smithi144 2024-06-30T13:06:14.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:14 smithi144 bash[19242]: cephadm 2024-06-30T13:06:12.725444+0000 mgr.y (mgr.14144) 92 : cephadm [INF] Deploying daemon osd.2 on smithi144 2024-06-30T13:06:14.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:14 smithi167 bash[14619]: cephadm 2024-06-30T13:06:12.725444+0000 mgr.y (mgr.14144) 92 : cephadm [INF] Deploying daemon osd.2 on smithi144 2024-06-30T13:06:15.439 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 2 on host 'smithi144' 2024-06-30T13:06:15.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:15 smithi144 bash[14758]: cluster 2024-06-30T13:06:14.330315+0000 mgr.y (mgr.14144) 93 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:15.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:15 smithi144 bash[19242]: cluster 2024-06-30T13:06:14.330315+0000 mgr.y (mgr.14144) 93 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:15.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:15 smithi167 bash[14619]: cluster 2024-06-30T13:06:14.330315+0000 mgr.y (mgr.14144) 93 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:16.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:16 smithi144 bash[19242]: audit 2024-06-30T13:06:15.425807+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:16.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:16 smithi144 bash[19242]: audit 2024-06-30T13:06:15.430344+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:16.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:16 smithi144 bash[19242]: audit 2024-06-30T13:06:15.437548+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:16.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:16 smithi144 bash[14758]: audit 2024-06-30T13:06:15.425807+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:16.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:16 smithi144 bash[14758]: audit 2024-06-30T13:06:15.430344+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:16.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:16 smithi144 bash[14758]: audit 2024-06-30T13:06:15.437548+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:16.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:16 smithi167 bash[14619]: audit 2024-06-30T13:06:15.425807+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:16.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:16 smithi167 bash[14619]: audit 2024-06-30T13:06:15.430344+0000 mon.a (mon.0) 267 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:16.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:16 smithi167 bash[14619]: audit 2024-06-30T13:06:15.437548+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:16.888 DEBUG:teuthology.orchestra.run.smithi144:osd.2> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.2.service 2024-06-30T13:06:16.890 INFO:tasks.cephadm:Deploying osd.3 on smithi144 with /dev/vg_nvme/lv_1... 2024-06-30T13:06:16.890 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-30T13:06:17.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:17 smithi144 bash[19242]: cluster 2024-06-30T13:06:16.330841+0000 mgr.y (mgr.14144) 94 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:17.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:17 smithi144 bash[14758]: cluster 2024-06-30T13:06:16.330841+0000 mgr.y (mgr.14144) 94 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:17.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:17 smithi167 bash[14619]: cluster 2024-06-30T13:06:16.330841+0000 mgr.y (mgr.14144) 94 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:17.928 INFO:journalctl@ceph.osd.2.smithi144.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:06:18.668 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:18 smithi144 bash[27047]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-30T13:06:18.668 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:18 smithi144 bash[27047]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2024-06-30T13:06:18.668 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:18 smithi144 bash[27047]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2/block 2024-06-30T13:06:18.668 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:18 smithi144 bash[27047]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-2/block 2024-06-30T13:06:18.668 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:18 smithi144 bash[27047]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-30T13:06:18.668 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:18 smithi144 bash[27047]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-06-30T13:06:18.668 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:18 smithi144 bash[27047]: --> ceph-volume lvm activate successful for osd ID: 2 2024-06-30T13:06:19.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:19 smithi144 bash[19242]: cluster 2024-06-30T13:06:18.331252+0000 mgr.y (mgr.14144) 95 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:19.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:19 smithi144 bash[19242]: audit 2024-06-30T13:06:18.718303+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:19.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:19 smithi144 bash[14758]: cluster 2024-06-30T13:06:18.331252+0000 mgr.y (mgr.14144) 95 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:19.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:19 smithi144 bash[14758]: audit 2024-06-30T13:06:18.718303+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:19.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:19 smithi167 bash[14619]: cluster 2024-06-30T13:06:18.331252+0000 mgr.y (mgr.14144) 95 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:19.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:19 smithi167 bash[14619]: audit 2024-06-30T13:06:18.718303+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:21.508 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:21 smithi144 bash[14758]: cluster 2024-06-30T13:06:20.331729+0000 mgr.y (mgr.14144) 96 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:21.508 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:21 smithi144 bash[19242]: cluster 2024-06-30T13:06:20.331729+0000 mgr.y (mgr.14144) 96 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:21.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:21 smithi167 bash[14619]: cluster 2024-06-30T13:06:20.331729+0000 mgr.y (mgr.14144) 96 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:22.590 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:22 smithi144 bash[27411]: debug 2024-06-30T13:06:22.373+0000 7faff4317700 1 -- 172.21.15.144:0/1574096548 <== mon.0 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557738d4e4e0 con 0x557738dc7000 2024-06-30T13:06:22.998 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:22 smithi144 bash[27411]: debug 2024-06-30T13:06:22.673+0000 7faffbda2080 -1 Falling back to public interface 2024-06-30T13:06:23.553 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-30T13:06:23.554 INFO:teuthology.orchestra.run.smithi144.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-30T13:06:23.554 INFO:teuthology.orchestra.run.smithi144.stderr: stderr: 10+0 records in 2024-06-30T13:06:23.554 INFO:teuthology.orchestra.run.smithi144.stderr:10+0 records out 2024-06-30T13:06:23.554 INFO:teuthology.orchestra.run.smithi144.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0151017 s, 694 MB/s 2024-06-30T13:06:23.554 INFO:teuthology.orchestra.run.smithi144.stderr:--> Zapping successful for: 2024-06-30T13:06:23.602 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:23 smithi144 bash[19242]: cluster 2024-06-30T13:06:22.332173+0000 mgr.y (mgr.14144) 97 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:23.602 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:23 smithi144 bash[14758]: cluster 2024-06-30T13:06:22.332173+0000 mgr.y (mgr.14144) 97 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:23.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:23 smithi167 bash[14619]: cluster 2024-06-30T13:06:22.332173+0000 mgr.y (mgr.14144) 97 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:24.647 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_1 2024-06-30T13:06:25.723 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:25 smithi144 bash[14758]: cluster 2024-06-30T13:06:24.332718+0000 mgr.y (mgr.14144) 98 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:25.723 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:25 smithi144 bash[19242]: cluster 2024-06-30T13:06:24.332718+0000 mgr.y (mgr.14144) 98 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:25.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:25 smithi167 bash[14619]: cluster 2024-06-30T13:06:24.332718+0000 mgr.y (mgr.14144) 98 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:26.008 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:25 smithi144 bash[27411]: debug 2024-06-30T13:06:25.717+0000 7faffbda2080 -1 osd.2 0 log_to_monitors {default=true} 2024-06-30T13:06:26.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:26 smithi167 bash[14619]: audit 2024-06-30T13:06:25.727767+0000 mon.a (mon.0) 270 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-30T13:06:26.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:26 smithi167 bash[14619]: cluster 2024-06-30T13:06:26.333218+0000 mgr.y (mgr.14144) 99 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:26.785 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:26 smithi144 bash[14758]: audit 2024-06-30T13:06:25.727767+0000 mon.a (mon.0) 270 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-30T13:06:26.785 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:26 smithi144 bash[14758]: cluster 2024-06-30T13:06:26.333218+0000 mgr.y (mgr.14144) 99 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:26.785 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:26 smithi144 bash[19242]: audit 2024-06-30T13:06:25.727767+0000 mon.a (mon.0) 270 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-30T13:06:26.785 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:26 smithi144 bash[19242]: cluster 2024-06-30T13:06:26.333218+0000 mgr.y (mgr.14144) 99 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:27.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:27 smithi144 bash[14758]: audit 2024-06-30T13:06:26.430369+0000 mon.a (mon.0) 271 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-30T13:06:27.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:27 smithi144 bash[14758]: cluster 2024-06-30T13:06:26.430465+0000 mon.a (mon.0) 272 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-06-30T13:06:27.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:27 smithi144 bash[14758]: audit 2024-06-30T13:06:26.430589+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:27.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:27 smithi144 bash[14758]: audit 2024-06-30T13:06:26.431419+0000 mon.a (mon.0) 274 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:06:27.759 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:06:27 smithi144 bash[27411]: debug 2024-06-30T13:06:27.437+0000 7faff2313700 -1 osd.2 0 waiting for initial osdmap 2024-06-30T13:06:27.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:27 smithi144 bash[19242]: audit 2024-06-30T13:06:26.430369+0000 mon.a (mon.0) 271 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-30T13:06:27.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:27 smithi144 bash[19242]: cluster 2024-06-30T13:06:26.430465+0000 mon.a (mon.0) 272 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-06-30T13:06:27.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:27 smithi144 bash[19242]: audit 2024-06-30T13:06:26.430589+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:27.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:27 smithi144 bash[19242]: audit 2024-06-30T13:06:26.431419+0000 mon.a (mon.0) 274 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:06:27.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:27 smithi167 bash[14619]: audit 2024-06-30T13:06:26.430369+0000 mon.a (mon.0) 271 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-30T13:06:27.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:27 smithi167 bash[14619]: cluster 2024-06-30T13:06:26.430465+0000 mon.a (mon.0) 272 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-06-30T13:06:27.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:27 smithi167 bash[14619]: audit 2024-06-30T13:06:26.430589+0000 mon.a (mon.0) 273 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:27.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:27 smithi167 bash[14619]: audit 2024-06-30T13:06:26.431419+0000 mon.a (mon.0) 274 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:06:29.258 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: audit 2024-06-30T13:06:27.433789+0000 mon.a (mon.0) 275 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:06:29.258 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: cluster 2024-06-30T13:06:27.434024+0000 mon.a (mon.0) 276 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: audit 2024-06-30T13:06:27.434890+0000 mon.a (mon.0) 277 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: audit 2024-06-30T13:06:27.444576+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: cephadm 2024-06-30T13:06:27.759303+0000 mgr.y (mgr.14144) 100 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: audit 2024-06-30T13:06:27.769001+0000 mon.a (mon.0) 279 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: audit 2024-06-30T13:06:27.776962+0000 mon.a (mon.0) 280 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: audit 2024-06-30T13:06:27.780184+0000 mon.a (mon.0) 281 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: audit 2024-06-30T13:06:28.050542+0000 mon.a (mon.0) 282 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[19242]: cluster 2024-06-30T13:06:28.333765+0000 mgr.y (mgr.14144) 101 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: audit 2024-06-30T13:06:27.433789+0000 mon.a (mon.0) 275 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: cluster 2024-06-30T13:06:27.434024+0000 mon.a (mon.0) 276 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: audit 2024-06-30T13:06:27.434890+0000 mon.a (mon.0) 277 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:29.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: audit 2024-06-30T13:06:27.444576+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:29.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: cephadm 2024-06-30T13:06:27.759303+0000 mgr.y (mgr.14144) 100 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:06:29.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: audit 2024-06-30T13:06:27.769001+0000 mon.a (mon.0) 279 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:29.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: audit 2024-06-30T13:06:27.776962+0000 mon.a (mon.0) 280 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:29.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: audit 2024-06-30T13:06:27.780184+0000 mon.a (mon.0) 281 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:29.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: audit 2024-06-30T13:06:28.050542+0000 mon.a (mon.0) 282 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:29.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:28 smithi144 bash[14758]: cluster 2024-06-30T13:06:28.333765+0000 mgr.y (mgr.14144) 101 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: audit 2024-06-30T13:06:27.433789+0000 mon.a (mon.0) 275 : audit [INF] from='osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: cluster 2024-06-30T13:06:27.434024+0000 mon.a (mon.0) 276 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: audit 2024-06-30T13:06:27.434890+0000 mon.a (mon.0) 277 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: audit 2024-06-30T13:06:27.444576+0000 mon.a (mon.0) 278 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: cephadm 2024-06-30T13:06:27.759303+0000 mgr.y (mgr.14144) 100 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: audit 2024-06-30T13:06:27.769001+0000 mon.a (mon.0) 279 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: audit 2024-06-30T13:06:27.776962+0000 mon.a (mon.0) 280 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: audit 2024-06-30T13:06:27.780184+0000 mon.a (mon.0) 281 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: audit 2024-06-30T13:06:28.050542+0000 mon.a (mon.0) 282 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:28 smithi167 bash[14619]: cluster 2024-06-30T13:06:28.333765+0000 mgr.y (mgr.14144) 101 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2024-06-30T13:06:30.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: cluster 2024-06-30T13:06:26.728852+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:06:30.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: cluster 2024-06-30T13:06:26.729117+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: cluster 2024-06-30T13:06:28.444444+0000 mon.a (mon.0) 283 : cluster [INF] osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724] boot 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: cluster 2024-06-30T13:06:28.444553+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: audit 2024-06-30T13:06:28.445401+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: audit 2024-06-30T13:06:28.445939+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: audit 2024-06-30T13:06:28.976178+0000 mgr.y (mgr.14144) 102 : audit [DBG] from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: audit 2024-06-30T13:06:29.445593+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: cluster 2024-06-30T13:06:29.445744+0000 mon.a (mon.0) 288 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[14758]: audit 2024-06-30T13:06:29.447558+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: cluster 2024-06-30T13:06:26.728852+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: cluster 2024-06-30T13:06:26.729117+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:06:30.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: cluster 2024-06-30T13:06:28.444444+0000 mon.a (mon.0) 283 : cluster [INF] osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724] boot 2024-06-30T13:06:30.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: cluster 2024-06-30T13:06:28.444553+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-06-30T13:06:30.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: audit 2024-06-30T13:06:28.445401+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:30.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: audit 2024-06-30T13:06:28.445939+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2024-06-30T13:06:30.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: audit 2024-06-30T13:06:28.976178+0000 mgr.y (mgr.14144) 102 : audit [DBG] from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:06:30.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: audit 2024-06-30T13:06:29.445593+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2024-06-30T13:06:30.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: cluster 2024-06-30T13:06:29.445744+0000 mon.a (mon.0) 288 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-06-30T13:06:30.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:29 smithi144 bash[19242]: audit 2024-06-30T13:06:29.447558+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: cluster 2024-06-30T13:06:26.728852+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: cluster 2024-06-30T13:06:26.729117+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: cluster 2024-06-30T13:06:28.444444+0000 mon.a (mon.0) 283 : cluster [INF] osd.2 [v2:172.21.15.144:6818/12822724,v1:172.21.15.144:6819/12822724] boot 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: cluster 2024-06-30T13:06:28.444553+0000 mon.a (mon.0) 284 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: audit 2024-06-30T13:06:28.445401+0000 mon.a (mon.0) 285 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: audit 2024-06-30T13:06:28.445939+0000 mon.a (mon.0) 286 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: audit 2024-06-30T13:06:28.976178+0000 mgr.y (mgr.14144) 102 : audit [DBG] from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: audit 2024-06-30T13:06:29.445593+0000 mon.a (mon.0) 287 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: cluster 2024-06-30T13:06:29.445744+0000 mon.a (mon.0) 288 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-06-30T13:06:30.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:29 smithi167 bash[14619]: audit 2024-06-30T13:06:29.447558+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-06-30T13:06:31.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[14758]: cluster 2024-06-30T13:06:30.334313+0000 mgr.y (mgr.14144) 103 : cluster [DBG] pgmap v86: 1 pgs: 1 unknown; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:31.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[14758]: audit 2024-06-30T13:06:30.448500+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-06-30T13:06:31.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[14758]: cluster 2024-06-30T13:06:30.448775+0000 mon.a (mon.0) 291 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-06-30T13:06:31.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[14758]: audit 2024-06-30T13:06:30.456701+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:06:31.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[14758]: audit 2024-06-30T13:06:30.458790+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:06:31.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[14758]: audit 2024-06-30T13:06:30.459878+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[19242]: cluster 2024-06-30T13:06:30.334313+0000 mgr.y (mgr.14144) 103 : cluster [DBG] pgmap v86: 1 pgs: 1 unknown; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[19242]: audit 2024-06-30T13:06:30.448500+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-06-30T13:06:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[19242]: cluster 2024-06-30T13:06:30.448775+0000 mon.a (mon.0) 291 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-06-30T13:06:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[19242]: audit 2024-06-30T13:06:30.456701+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:06:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[19242]: audit 2024-06-30T13:06:30.458790+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:06:31.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:30 smithi144 bash[19242]: audit 2024-06-30T13:06:30.459878+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:31.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:30 smithi167 bash[14619]: cluster 2024-06-30T13:06:30.334313+0000 mgr.y (mgr.14144) 103 : cluster [DBG] pgmap v86: 1 pgs: 1 unknown; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:31.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:30 smithi167 bash[14619]: audit 2024-06-30T13:06:30.448500+0000 mon.a (mon.0) 290 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-06-30T13:06:31.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:30 smithi167 bash[14619]: cluster 2024-06-30T13:06:30.448775+0000 mon.a (mon.0) 291 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-06-30T13:06:31.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:30 smithi167 bash[14619]: audit 2024-06-30T13:06:30.456701+0000 mon.a (mon.0) 292 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:06:31.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:30 smithi167 bash[14619]: audit 2024-06-30T13:06:30.458790+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:06:31.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:30 smithi167 bash[14619]: audit 2024-06-30T13:06:30.459878+0000 mon.a (mon.0) 294 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:32.205 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:31 smithi144 bash[14758]: cephadm 2024-06-30T13:06:30.458448+0000 mgr.y (mgr.14144) 104 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:06:32.205 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:31 smithi144 bash[14758]: cluster 2024-06-30T13:06:31.452660+0000 mon.a (mon.0) 295 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-06-30T13:06:32.205 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:31 smithi144 bash[19242]: cephadm 2024-06-30T13:06:30.458448+0000 mgr.y (mgr.14144) 104 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:06:32.205 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:31 smithi144 bash[19242]: cluster 2024-06-30T13:06:31.452660+0000 mon.a (mon.0) 295 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-06-30T13:06:32.276 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:31 smithi167 bash[14619]: cephadm 2024-06-30T13:06:30.458448+0000 mgr.y (mgr.14144) 104 : cephadm [INF] Applying service osd.None on host smithi144... 2024-06-30T13:06:32.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:31 smithi167 bash[14619]: cluster 2024-06-30T13:06:31.452660+0000 mon.a (mon.0) 295 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-06-30T13:06:33.258 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:32 smithi144 bash[19242]: cluster 2024-06-30T13:06:32.334856+0000 mgr.y (mgr.14144) 105 : cluster [DBG] pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:33.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:32 smithi144 bash[14758]: cluster 2024-06-30T13:06:32.334856+0000 mgr.y (mgr.14144) 105 : cluster [DBG] pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:33.276 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:32 smithi167 bash[14619]: cluster 2024-06-30T13:06:32.334856+0000 mgr.y (mgr.14144) 105 : cluster [DBG] pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:35.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[14758]: cluster 2024-06-30T13:06:34.335461+0000 mgr.y (mgr.14144) 106 : cluster [DBG] pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:35.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[14758]: audit 2024-06-30T13:06:34.693612+0000 mon.c (mon.2) 5 : audit [INF] from='client.? 172.21.15.144:0/1663587307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6529f96-89af-4874-92c0-6bf4cf3ee6b4"}]: dispatch 2024-06-30T13:06:35.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[14758]: audit 2024-06-30T13:06:34.694256+0000 mon.a (mon.0) 296 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6529f96-89af-4874-92c0-6bf4cf3ee6b4"}]: dispatch 2024-06-30T13:06:35.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[14758]: audit 2024-06-30T13:06:34.707102+0000 mon.a (mon.0) 297 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6529f96-89af-4874-92c0-6bf4cf3ee6b4"}]': finished 2024-06-30T13:06:35.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[14758]: cluster 2024-06-30T13:06:34.707332+0000 mon.a (mon.0) 298 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-06-30T13:06:35.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[14758]: audit 2024-06-30T13:06:34.707629+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:35.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[14758]: audit 2024-06-30T13:06:35.369497+0000 mon.a (mon.0) 300 : audit [DBG] from='client.? 172.21.15.144:0/1582131544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:06:35.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[19242]: cluster 2024-06-30T13:06:34.335461+0000 mgr.y (mgr.14144) 106 : cluster [DBG] pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:35.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[19242]: audit 2024-06-30T13:06:34.693612+0000 mon.c (mon.2) 5 : audit [INF] from='client.? 172.21.15.144:0/1663587307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6529f96-89af-4874-92c0-6bf4cf3ee6b4"}]: dispatch 2024-06-30T13:06:35.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[19242]: audit 2024-06-30T13:06:34.694256+0000 mon.a (mon.0) 296 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6529f96-89af-4874-92c0-6bf4cf3ee6b4"}]: dispatch 2024-06-30T13:06:35.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[19242]: audit 2024-06-30T13:06:34.707102+0000 mon.a (mon.0) 297 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6529f96-89af-4874-92c0-6bf4cf3ee6b4"}]': finished 2024-06-30T13:06:35.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[19242]: cluster 2024-06-30T13:06:34.707332+0000 mon.a (mon.0) 298 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-06-30T13:06:35.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[19242]: audit 2024-06-30T13:06:34.707629+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:35.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:35 smithi144 bash[19242]: audit 2024-06-30T13:06:35.369497+0000 mon.a (mon.0) 300 : audit [DBG] from='client.? 172.21.15.144:0/1582131544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:06:35.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:35 smithi167 bash[14619]: cluster 2024-06-30T13:06:34.335461+0000 mgr.y (mgr.14144) 106 : cluster [DBG] pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:35.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:35 smithi167 bash[14619]: audit 2024-06-30T13:06:34.693612+0000 mon.c (mon.2) 5 : audit [INF] from='client.? 172.21.15.144:0/1663587307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6529f96-89af-4874-92c0-6bf4cf3ee6b4"}]: dispatch 2024-06-30T13:06:35.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:35 smithi167 bash[14619]: audit 2024-06-30T13:06:34.694256+0000 mon.a (mon.0) 296 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f6529f96-89af-4874-92c0-6bf4cf3ee6b4"}]: dispatch 2024-06-30T13:06:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:35 smithi167 bash[14619]: audit 2024-06-30T13:06:34.707102+0000 mon.a (mon.0) 297 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6529f96-89af-4874-92c0-6bf4cf3ee6b4"}]': finished 2024-06-30T13:06:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:35 smithi167 bash[14619]: cluster 2024-06-30T13:06:34.707332+0000 mon.a (mon.0) 298 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-06-30T13:06:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:35 smithi167 bash[14619]: audit 2024-06-30T13:06:34.707629+0000 mon.a (mon.0) 299 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:35 smithi167 bash[14619]: audit 2024-06-30T13:06:35.369497+0000 mon.a (mon.0) 300 : audit [DBG] from='client.? 172.21.15.144:0/1582131544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:06:37.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:37 smithi144 bash[19242]: cluster 2024-06-30T13:06:36.335935+0000 mgr.y (mgr.14144) 107 : cluster [DBG] pgmap v92: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:37.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:37 smithi144 bash[14758]: cluster 2024-06-30T13:06:36.335935+0000 mgr.y (mgr.14144) 107 : cluster [DBG] pgmap v92: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:37.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:37 smithi167 bash[14619]: cluster 2024-06-30T13:06:36.335935+0000 mgr.y (mgr.14144) 107 : cluster [DBG] pgmap v92: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:39.704 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:39 smithi144 bash[19242]: cluster 2024-06-30T13:06:38.336522+0000 mgr.y (mgr.14144) 108 : cluster [DBG] pgmap v93: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:39.704 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:39 smithi144 bash[14758]: cluster 2024-06-30T13:06:38.336522+0000 mgr.y (mgr.14144) 108 : cluster [DBG] pgmap v93: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:39.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:39 smithi167 bash[14619]: cluster 2024-06-30T13:06:38.336522+0000 mgr.y (mgr.14144) 108 : cluster [DBG] pgmap v93: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:41.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:41 smithi144 bash[19242]: cluster 2024-06-30T13:06:40.337160+0000 mgr.y (mgr.14144) 109 : cluster [DBG] pgmap v94: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:41.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:41 smithi144 bash[14758]: cluster 2024-06-30T13:06:40.337160+0000 mgr.y (mgr.14144) 109 : cluster [DBG] pgmap v94: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:41.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:41 smithi167 bash[14619]: cluster 2024-06-30T13:06:40.337160+0000 mgr.y (mgr.14144) 109 : cluster [DBG] pgmap v94: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:43.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:43 smithi144 bash[14758]: cluster 2024-06-30T13:06:42.337751+0000 mgr.y (mgr.14144) 110 : cluster [DBG] pgmap v95: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:43.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:43 smithi144 bash[14758]: audit 2024-06-30T13:06:42.838439+0000 mon.a (mon.0) 301 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-30T13:06:43.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:43 smithi144 bash[14758]: audit 2024-06-30T13:06:42.840203+0000 mon.a (mon.0) 302 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:43.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:43 smithi144 bash[19242]: cluster 2024-06-30T13:06:42.337751+0000 mgr.y (mgr.14144) 110 : cluster [DBG] pgmap v95: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:43.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:43 smithi144 bash[19242]: audit 2024-06-30T13:06:42.838439+0000 mon.a (mon.0) 301 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-30T13:06:43.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:43 smithi144 bash[19242]: audit 2024-06-30T13:06:42.840203+0000 mon.a (mon.0) 302 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:43.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:43 smithi167 bash[14619]: cluster 2024-06-30T13:06:42.337751+0000 mgr.y (mgr.14144) 110 : cluster [DBG] pgmap v95: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:43.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:43 smithi167 bash[14619]: audit 2024-06-30T13:06:42.838439+0000 mon.a (mon.0) 301 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-30T13:06:43.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:43 smithi167 bash[14619]: audit 2024-06-30T13:06:42.840203+0000 mon.a (mon.0) 302 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:44.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:44 smithi144 bash[14758]: cephadm 2024-06-30T13:06:42.841628+0000 mgr.y (mgr.14144) 111 : cephadm [INF] Deploying daemon osd.3 on smithi144 2024-06-30T13:06:44.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:44 smithi144 bash[19242]: cephadm 2024-06-30T13:06:42.841628+0000 mgr.y (mgr.14144) 111 : cephadm [INF] Deploying daemon osd.3 on smithi144 2024-06-30T13:06:44.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:44 smithi167 bash[14619]: cephadm 2024-06-30T13:06:42.841628+0000 mgr.y (mgr.14144) 111 : cephadm [INF] Deploying daemon osd.3 on smithi144 2024-06-30T13:06:45.475 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 3 on host 'smithi144' 2024-06-30T13:06:45.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:45 smithi144 bash[14758]: cluster 2024-06-30T13:06:44.338187+0000 mgr.y (mgr.14144) 112 : cluster [DBG] pgmap v96: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:45.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:45 smithi144 bash[19242]: cluster 2024-06-30T13:06:44.338187+0000 mgr.y (mgr.14144) 112 : cluster [DBG] pgmap v96: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:45.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:45 smithi167 bash[14619]: cluster 2024-06-30T13:06:44.338187+0000 mgr.y (mgr.14144) 112 : cluster [DBG] pgmap v96: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:46.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:46 smithi144 bash[14758]: audit 2024-06-30T13:06:45.461617+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:46.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:46 smithi144 bash[14758]: audit 2024-06-30T13:06:45.467109+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:46 smithi144 bash[14758]: audit 2024-06-30T13:06:45.474120+0000 mon.a (mon.0) 305 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:46 smithi144 bash[14758]: cluster 2024-06-30T13:06:46.338696+0000 mgr.y (mgr.14144) 113 : cluster [DBG] pgmap v97: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:46.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:46 smithi144 bash[19242]: audit 2024-06-30T13:06:45.461617+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:46.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:46 smithi144 bash[19242]: audit 2024-06-30T13:06:45.467109+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:46.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:46 smithi144 bash[19242]: audit 2024-06-30T13:06:45.474120+0000 mon.a (mon.0) 305 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:46.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:46 smithi144 bash[19242]: cluster 2024-06-30T13:06:46.338696+0000 mgr.y (mgr.14144) 113 : cluster [DBG] pgmap v97: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:46.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:46 smithi167 bash[14619]: audit 2024-06-30T13:06:45.461617+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:46.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:46 smithi167 bash[14619]: audit 2024-06-30T13:06:45.467109+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:46.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:46 smithi167 bash[14619]: audit 2024-06-30T13:06:45.474120+0000 mon.a (mon.0) 305 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:46.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:46 smithi167 bash[14619]: cluster 2024-06-30T13:06:46.338696+0000 mgr.y (mgr.14144) 113 : cluster [DBG] pgmap v97: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:46.884 DEBUG:teuthology.orchestra.run.smithi144:osd.3> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.3.service 2024-06-30T13:06:46.886 INFO:tasks.cephadm:Deploying osd.4 on smithi167 with /dev/vg_nvme/lv_4... 2024-06-30T13:06:46.887 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-30T13:06:47.241 INFO:journalctl@ceph.osd.3.smithi144.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:06:48.258 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:47 smithi144 bash[29801]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-30T13:06:48.258 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:47 smithi144 bash[29801]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_1 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2024-06-30T13:06:48.258 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:47 smithi144 bash[29801]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3/block 2024-06-30T13:06:48.259 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:47 smithi144 bash[29801]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-3/block 2024-06-30T13:06:48.259 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:47 smithi144 bash[29801]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-06-30T13:06:48.259 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:47 smithi144 bash[29801]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-06-30T13:06:48.259 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:47 smithi144 bash[29801]: --> ceph-volume lvm activate successful for osd ID: 3 2024-06-30T13:06:49.396 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:49 smithi144 bash[14758]: cluster 2024-06-30T13:06:48.339299+0000 mgr.y (mgr.14144) 114 : cluster [DBG] pgmap v98: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:49.679 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:49 smithi167 bash[14619]: cluster 2024-06-30T13:06:48.339299+0000 mgr.y (mgr.14144) 114 : cluster [DBG] pgmap v98: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:49.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:49 smithi144 bash[19242]: cluster 2024-06-30T13:06:48.339299+0000 mgr.y (mgr.14144) 114 : cluster [DBG] pgmap v98: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:50.231 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-30T13:06:50.231 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-30T13:06:50.231 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-30T13:06:50.231 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-30T13:06:50.231 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0152971 s, 685 MB/s 2024-06-30T13:06:50.231 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-30T13:06:50.258 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:50 smithi144 bash[29945]: debug 2024-06-30T13:06:50.184+0000 7f8a01702080 -1 Falling back to public interface 2024-06-30T13:06:50.862 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-06-30T13:06:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:50 smithi144 bash[14758]: audit 2024-06-30T13:06:49.723536+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:51.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:50 smithi144 bash[14758]: cluster 2024-06-30T13:06:50.339834+0000 mgr.y (mgr.14144) 115 : cluster [DBG] pgmap v99: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:51.008 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:50 smithi144 bash[19242]: audit 2024-06-30T13:06:49.723536+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:51.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:50 smithi144 bash[19242]: cluster 2024-06-30T13:06:50.339834+0000 mgr.y (mgr.14144) 115 : cluster [DBG] pgmap v99: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:51.026 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:50 smithi167 bash[14619]: audit 2024-06-30T13:06:49.723536+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:51.026 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:50 smithi167 bash[14619]: cluster 2024-06-30T13:06:50.339834+0000 mgr.y (mgr.14144) 115 : cluster [DBG] pgmap v99: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:53.406 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:53 smithi144 bash[29945]: debug 2024-06-30T13:06:53.164+0000 7f8a01702080 -1 osd.3 0 log_to_monitors {default=true} 2024-06-30T13:06:53.408 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:53 smithi144 bash[14758]: cluster 2024-06-30T13:06:52.340395+0000 mgr.y (mgr.14144) 116 : cluster [DBG] pgmap v100: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:53.409 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:53 smithi144 bash[14758]: audit 2024-06-30T13:06:53.171360+0000 mon.c (mon.2) 6 : audit [INF] from='osd.3 [v2:172.21.15.144:6826/22475720,v1:172.21.15.144:6827/22475720]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-30T13:06:53.409 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:53 smithi144 bash[14758]: audit 2024-06-30T13:06:53.171859+0000 mon.a (mon.0) 307 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-30T13:06:53.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:53 smithi144 bash[19242]: cluster 2024-06-30T13:06:52.340395+0000 mgr.y (mgr.14144) 116 : cluster [DBG] pgmap v100: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:53.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:53 smithi144 bash[19242]: audit 2024-06-30T13:06:53.171360+0000 mon.c (mon.2) 6 : audit [INF] from='osd.3 [v2:172.21.15.144:6826/22475720,v1:172.21.15.144:6827/22475720]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-30T13:06:53.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:53 smithi144 bash[19242]: audit 2024-06-30T13:06:53.171859+0000 mon.a (mon.0) 307 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-30T13:06:53.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:53 smithi167 bash[14619]: cluster 2024-06-30T13:06:52.340395+0000 mgr.y (mgr.14144) 116 : cluster [DBG] pgmap v100: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:53.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:53 smithi167 bash[14619]: audit 2024-06-30T13:06:53.171360+0000 mon.c (mon.2) 6 : audit [INF] from='osd.3 [v2:172.21.15.144:6826/22475720,v1:172.21.15.144:6827/22475720]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-30T13:06:53.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:53 smithi167 bash[14619]: audit 2024-06-30T13:06:53.171859+0000 mon.a (mon.0) 307 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-30T13:06:54.699 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:54 smithi167 bash[14619]: audit 2024-06-30T13:06:53.401668+0000 mon.a (mon.0) 308 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-30T13:06:54.699 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:54 smithi167 bash[14619]: cluster 2024-06-30T13:06:53.401840+0000 mon.a (mon.0) 309 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-06-30T13:06:54.699 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:54 smithi167 bash[14619]: audit 2024-06-30T13:06:53.402493+0000 mon.a (mon.0) 310 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:54.700 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:54 smithi167 bash[14619]: audit 2024-06-30T13:06:53.404049+0000 mon.c (mon.2) 7 : audit [INF] from='osd.3 [v2:172.21.15.144:6826/22475720,v1:172.21.15.144:6827/22475720]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:06:54.700 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:54 smithi167 bash[14619]: audit 2024-06-30T13:06:53.409045+0000 mon.a (mon.0) 311 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:06:54.700 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:54 smithi167 bash[14619]: audit 2024-06-30T13:06:53.893669+0000 mon.a (mon.0) 312 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:06:54.700 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:54 smithi167 bash[14619]: audit 2024-06-30T13:06:53.897812+0000 mon.a (mon.0) 313 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:06:54.700 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:54 smithi167 bash[14619]: audit 2024-06-30T13:06:53.899695+0000 mon.a (mon.0) 314 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:54.758 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[29945]: debug 2024-06-30T13:06:54.420+0000 7f89f7c73700 -1 osd.3 0 waiting for initial osdmap 2024-06-30T13:06:54.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[19242]: audit 2024-06-30T13:06:53.401668+0000 mon.a (mon.0) 308 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-30T13:06:54.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[19242]: cluster 2024-06-30T13:06:53.401840+0000 mon.a (mon.0) 309 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[19242]: audit 2024-06-30T13:06:53.402493+0000 mon.a (mon.0) 310 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[19242]: audit 2024-06-30T13:06:53.404049+0000 mon.c (mon.2) 7 : audit [INF] from='osd.3 [v2:172.21.15.144:6826/22475720,v1:172.21.15.144:6827/22475720]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[19242]: audit 2024-06-30T13:06:53.409045+0000 mon.a (mon.0) 311 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[19242]: audit 2024-06-30T13:06:53.893669+0000 mon.a (mon.0) 312 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[19242]: audit 2024-06-30T13:06:53.897812+0000 mon.a (mon.0) 313 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[19242]: audit 2024-06-30T13:06:53.899695+0000 mon.a (mon.0) 314 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[14758]: audit 2024-06-30T13:06:53.401668+0000 mon.a (mon.0) 308 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[14758]: cluster 2024-06-30T13:06:53.401840+0000 mon.a (mon.0) 309 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[14758]: audit 2024-06-30T13:06:53.402493+0000 mon.a (mon.0) 310 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[14758]: audit 2024-06-30T13:06:53.404049+0000 mon.c (mon.2) 7 : audit [INF] from='osd.3 [v2:172.21.15.144:6826/22475720,v1:172.21.15.144:6827/22475720]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[14758]: audit 2024-06-30T13:06:53.409045+0000 mon.a (mon.0) 311 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-06-30T13:06:54.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[14758]: audit 2024-06-30T13:06:53.893669+0000 mon.a (mon.0) 312 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:06:54.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[14758]: audit 2024-06-30T13:06:53.897812+0000 mon.a (mon.0) 313 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:06:54.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:54 smithi144 bash[14758]: audit 2024-06-30T13:06:53.899695+0000 mon.a (mon.0) 314 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:06:55.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[14758]: audit 2024-06-30T13:06:53.891887+0000 mgr.y (mgr.14144) 117 : audit [DBG] from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:06:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[14758]: cephadm 2024-06-30T13:06:53.897260+0000 mgr.y (mgr.14144) 118 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:06:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[14758]: cluster 2024-06-30T13:06:54.340961+0000 mgr.y (mgr.14144) 119 : cluster [DBG] pgmap v102: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[14758]: audit 2024-06-30T13:06:54.419688+0000 mon.a (mon.0) 315 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:06:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[14758]: cluster 2024-06-30T13:06:54.419836+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-06-30T13:06:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[14758]: audit 2024-06-30T13:06:54.420019+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[14758]: audit 2024-06-30T13:06:54.424303+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[14758]: audit 2024-06-30T13:06:55.301039+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:55.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[19242]: audit 2024-06-30T13:06:53.891887+0000 mgr.y (mgr.14144) 117 : audit [DBG] from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:06:55.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[19242]: cephadm 2024-06-30T13:06:53.897260+0000 mgr.y (mgr.14144) 118 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:06:55.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[19242]: cluster 2024-06-30T13:06:54.340961+0000 mgr.y (mgr.14144) 119 : cluster [DBG] pgmap v102: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:55.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[19242]: audit 2024-06-30T13:06:54.419688+0000 mon.a (mon.0) 315 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:06:55.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[19242]: cluster 2024-06-30T13:06:54.419836+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-06-30T13:06:55.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[19242]: audit 2024-06-30T13:06:54.420019+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:55.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[19242]: audit 2024-06-30T13:06:54.424303+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:55.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:55 smithi144 bash[19242]: audit 2024-06-30T13:06:55.301039+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:55.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:55 smithi167 bash[14619]: audit 2024-06-30T13:06:53.891887+0000 mgr.y (mgr.14144) 117 : audit [DBG] from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:06:55.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:55 smithi167 bash[14619]: cephadm 2024-06-30T13:06:53.897260+0000 mgr.y (mgr.14144) 118 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:06:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:55 smithi167 bash[14619]: cluster 2024-06-30T13:06:54.340961+0000 mgr.y (mgr.14144) 119 : cluster [DBG] pgmap v102: 1 pgs: 1 active+clean; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2024-06-30T13:06:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:55 smithi167 bash[14619]: audit 2024-06-30T13:06:54.419688+0000 mon.a (mon.0) 315 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-06-30T13:06:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:55 smithi167 bash[14619]: cluster 2024-06-30T13:06:54.419836+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-06-30T13:06:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:55 smithi167 bash[14619]: audit 2024-06-30T13:06:54.420019+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:55 smithi167 bash[14619]: audit 2024-06-30T13:06:54.424303+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:55 smithi167 bash[14619]: audit 2024-06-30T13:06:55.301039+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:56.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: cluster 2024-06-30T13:06:54.143583+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:06:56.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: cluster 2024-06-30T13:06:54.143792+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:06:56.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: cephadm 2024-06-30T13:06:55.291908+0000 mgr.y (mgr.14144) 120 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:06:56.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: cluster 2024-06-30T13:06:55.424067+0000 mon.a (mon.0) 320 : cluster [INF] osd.3 [v2:172.21.15.144:6826/22475720,v1:172.21.15.144:6827/22475720] boot 2024-06-30T13:06:56.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: cluster 2024-06-30T13:06:55.424183+0000 mon.a (mon.0) 321 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-06-30T13:06:56.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: audit 2024-06-30T13:06:55.425050+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:56.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: audit 2024-06-30T13:06:55.540354+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:56.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: audit 2024-06-30T13:06:55.543193+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: audit 2024-06-30T13:06:55.550600+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[14758]: cluster 2024-06-30T13:06:56.341522+0000 mgr.y (mgr.14144) 121 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 19 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: cluster 2024-06-30T13:06:54.143583+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: cluster 2024-06-30T13:06:54.143792+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: cephadm 2024-06-30T13:06:55.291908+0000 mgr.y (mgr.14144) 120 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: cluster 2024-06-30T13:06:55.424067+0000 mon.a (mon.0) 320 : cluster [INF] osd.3 [v2:172.21.15.144:6826/22475720,v1:172.21.15.144:6827/22475720] boot 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: cluster 2024-06-30T13:06:55.424183+0000 mon.a (mon.0) 321 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: audit 2024-06-30T13:06:55.425050+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: audit 2024-06-30T13:06:55.540354+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:56.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: audit 2024-06-30T13:06:55.543193+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:56.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: audit 2024-06-30T13:06:55.550600+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:56.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:56 smithi144 bash[19242]: cluster 2024-06-30T13:06:56.341522+0000 mgr.y (mgr.14144) 121 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 19 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:06:56.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: cluster 2024-06-30T13:06:54.143583+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:06:56.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: cluster 2024-06-30T13:06:54.143792+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:06:56.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: cephadm 2024-06-30T13:06:55.291908+0000 mgr.y (mgr.14144) 120 : cephadm [INF] Detected new or changed devices on smithi144 2024-06-30T13:06:56.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: cluster 2024-06-30T13:06:55.424067+0000 mon.a (mon.0) 320 : cluster [INF] osd.3 [v2:172.21.15.144:6826/22475720,v1:172.21.15.144:6827/22475720] boot 2024-06-30T13:06:56.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: cluster 2024-06-30T13:06:55.424183+0000 mon.a (mon.0) 321 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-06-30T13:06:56.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: audit 2024-06-30T13:06:55.425050+0000 mon.a (mon.0) 322 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-30T13:06:56.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: audit 2024-06-30T13:06:55.540354+0000 mon.a (mon.0) 323 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:56.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: audit 2024-06-30T13:06:55.543193+0000 mon.a (mon.0) 324 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:06:56.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: audit 2024-06-30T13:06:55.550600+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:06:56.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:56 smithi167 bash[14619]: cluster 2024-06-30T13:06:56.341522+0000 mgr.y (mgr.14144) 121 : cluster [DBG] pgmap v105: 1 pgs: 1 active+clean; 0 B data, 19 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:06:57.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:57 smithi144 bash[14758]: audit 2024-06-30T13:06:56.502207+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:06:57.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:57 smithi144 bash[14758]: audit 2024-06-30T13:06:56.504288+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:06:57.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:57 smithi144 bash[14758]: cluster 2024-06-30T13:06:56.560232+0000 mon.a (mon.0) 328 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-06-30T13:06:57.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:57 smithi144 bash[19242]: audit 2024-06-30T13:06:56.502207+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:06:57.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:57 smithi144 bash[19242]: audit 2024-06-30T13:06:56.504288+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:06:57.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:57 smithi144 bash[19242]: cluster 2024-06-30T13:06:56.560232+0000 mon.a (mon.0) 328 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-06-30T13:06:57.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:57 smithi167 bash[14619]: audit 2024-06-30T13:06:56.502207+0000 mon.a (mon.0) 326 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:06:57.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:57 smithi167 bash[14619]: audit 2024-06-30T13:06:56.504288+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:06:57.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:57 smithi167 bash[14619]: cluster 2024-06-30T13:06:56.560232+0000 mon.a (mon.0) 328 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-06-30T13:06:58.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[14758]: audit 2024-06-30T13:06:57.837169+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.167:0/80373622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c3e70d8-39f6-4dd5-96e5-840e20a18e04"}]: dispatch 2024-06-30T13:06:58.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[14758]: audit 2024-06-30T13:06:57.837274+0000 mon.a (mon.0) 329 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c3e70d8-39f6-4dd5-96e5-840e20a18e04"}]: dispatch 2024-06-30T13:06:58.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[14758]: audit 2024-06-30T13:06:57.845659+0000 mon.a (mon.0) 330 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c3e70d8-39f6-4dd5-96e5-840e20a18e04"}]': finished 2024-06-30T13:06:58.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[14758]: cluster 2024-06-30T13:06:57.845883+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-06-30T13:06:58.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[14758]: audit 2024-06-30T13:06:57.846267+0000 mon.a (mon.0) 332 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:06:58.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[14758]: cluster 2024-06-30T13:06:58.342109+0000 mgr.y (mgr.14144) 122 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:06:58.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[19242]: audit 2024-06-30T13:06:57.837169+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.167:0/80373622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c3e70d8-39f6-4dd5-96e5-840e20a18e04"}]: dispatch 2024-06-30T13:06:58.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[19242]: audit 2024-06-30T13:06:57.837274+0000 mon.a (mon.0) 329 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c3e70d8-39f6-4dd5-96e5-840e20a18e04"}]: dispatch 2024-06-30T13:06:58.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[19242]: audit 2024-06-30T13:06:57.845659+0000 mon.a (mon.0) 330 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c3e70d8-39f6-4dd5-96e5-840e20a18e04"}]': finished 2024-06-30T13:06:58.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[19242]: cluster 2024-06-30T13:06:57.845883+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-06-30T13:06:58.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[19242]: audit 2024-06-30T13:06:57.846267+0000 mon.a (mon.0) 332 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:06:58.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:58 smithi144 bash[19242]: cluster 2024-06-30T13:06:58.342109+0000 mgr.y (mgr.14144) 122 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:06:58.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:58 smithi167 bash[14619]: audit 2024-06-30T13:06:57.837169+0000 mon.b (mon.1) 7 : audit [INF] from='client.? 172.21.15.167:0/80373622' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c3e70d8-39f6-4dd5-96e5-840e20a18e04"}]: dispatch 2024-06-30T13:06:58.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:58 smithi167 bash[14619]: audit 2024-06-30T13:06:57.837274+0000 mon.a (mon.0) 329 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c3e70d8-39f6-4dd5-96e5-840e20a18e04"}]: dispatch 2024-06-30T13:06:58.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:58 smithi167 bash[14619]: audit 2024-06-30T13:06:57.845659+0000 mon.a (mon.0) 330 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c3e70d8-39f6-4dd5-96e5-840e20a18e04"}]': finished 2024-06-30T13:06:58.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:58 smithi167 bash[14619]: cluster 2024-06-30T13:06:57.845883+0000 mon.a (mon.0) 331 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-06-30T13:06:58.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:58 smithi167 bash[14619]: audit 2024-06-30T13:06:57.846267+0000 mon.a (mon.0) 332 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:06:58.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:58 smithi167 bash[14619]: cluster 2024-06-30T13:06:58.342109+0000 mgr.y (mgr.14144) 122 : cluster [DBG] pgmap v108: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:06:59.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:06:59 smithi144 bash[14758]: audit 2024-06-30T13:06:58.460913+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.167:0/3632102106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:06:59.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:06:59 smithi144 bash[19242]: audit 2024-06-30T13:06:58.460913+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.167:0/3632102106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:06:59.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:06:59 smithi167 bash[14619]: audit 2024-06-30T13:06:58.460913+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? 172.21.15.167:0/3632102106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:07:00.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:00 smithi144 bash[14758]: cluster 2024-06-30T13:07:00.342656+0000 mgr.y (mgr.14144) 123 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:00.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:00 smithi144 bash[19242]: cluster 2024-06-30T13:07:00.342656+0000 mgr.y (mgr.14144) 123 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:00.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:00 smithi167 bash[14619]: cluster 2024-06-30T13:07:00.342656+0000 mgr.y (mgr.14144) 123 : cluster [DBG] pgmap v109: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:03.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:03 smithi144 bash[14758]: cluster 2024-06-30T13:07:02.343246+0000 mgr.y (mgr.14144) 124 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:03.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:03 smithi144 bash[19242]: cluster 2024-06-30T13:07:02.343246+0000 mgr.y (mgr.14144) 124 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:03.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:03 smithi167 bash[14619]: cluster 2024-06-30T13:07:02.343246+0000 mgr.y (mgr.14144) 124 : cluster [DBG] pgmap v110: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:05.709 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:05 smithi167 bash[14619]: cluster 2024-06-30T13:07:04.343806+0000 mgr.y (mgr.14144) 125 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:05.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:05 smithi144 bash[14758]: cluster 2024-06-30T13:07:04.343806+0000 mgr.y (mgr.14144) 125 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:05.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:05 smithi144 bash[19242]: cluster 2024-06-30T13:07:04.343806+0000 mgr.y (mgr.14144) 125 : cluster [DBG] pgmap v111: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:06.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:06 smithi144 bash[14758]: audit 2024-06-30T13:07:05.414561+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-30T13:07:06.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:06 smithi144 bash[14758]: audit 2024-06-30T13:07:05.416497+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:06.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:06 smithi144 bash[14758]: cephadm 2024-06-30T13:07:05.417725+0000 mgr.y (mgr.14144) 126 : cephadm [INF] Deploying daemon osd.4 on smithi167 2024-06-30T13:07:06.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:06 smithi144 bash[19242]: audit 2024-06-30T13:07:05.414561+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-30T13:07:06.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:06 smithi144 bash[19242]: audit 2024-06-30T13:07:05.416497+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:06.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:06 smithi144 bash[19242]: cephadm 2024-06-30T13:07:05.417725+0000 mgr.y (mgr.14144) 126 : cephadm [INF] Deploying daemon osd.4 on smithi167 2024-06-30T13:07:06.764 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:06 smithi167 bash[14619]: audit 2024-06-30T13:07:05.414561+0000 mon.a (mon.0) 333 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-30T13:07:06.764 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:06 smithi167 bash[14619]: audit 2024-06-30T13:07:05.416497+0000 mon.a (mon.0) 334 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:06.764 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:06 smithi167 bash[14619]: cephadm 2024-06-30T13:07:05.417725+0000 mgr.y (mgr.14144) 126 : cephadm [INF] Deploying daemon osd.4 on smithi167 2024-06-30T13:07:07.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:07 smithi144 bash[14758]: cluster 2024-06-30T13:07:06.344348+0000 mgr.y (mgr.14144) 127 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:07.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:07 smithi144 bash[19242]: cluster 2024-06-30T13:07:06.344348+0000 mgr.y (mgr.14144) 127 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:07.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:07 smithi167 bash[14619]: cluster 2024-06-30T13:07:06.344348+0000 mgr.y (mgr.14144) 127 : cluster [DBG] pgmap v112: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:07.861 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 4 on host 'smithi167' 2024-06-30T13:07:08.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:08 smithi144 bash[14758]: audit 2024-06-30T13:07:07.840071+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:08.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:08 smithi144 bash[14758]: audit 2024-06-30T13:07:07.847847+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:08.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:08 smithi144 bash[14758]: audit 2024-06-30T13:07:07.858590+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:08.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:08 smithi144 bash[19242]: audit 2024-06-30T13:07:07.840071+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:08.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:08 smithi144 bash[19242]: audit 2024-06-30T13:07:07.847847+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:08.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:08 smithi144 bash[19242]: audit 2024-06-30T13:07:07.858590+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:08.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:08 smithi167 bash[14619]: audit 2024-06-30T13:07:07.840071+0000 mon.a (mon.0) 335 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:08.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:08 smithi167 bash[14619]: audit 2024-06-30T13:07:07.847847+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:08.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:08 smithi167 bash[14619]: audit 2024-06-30T13:07:07.858590+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:09.138 DEBUG:teuthology.orchestra.run.smithi167:osd.4> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.4.service 2024-06-30T13:07:09.140 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-06-30T13:07:09.140 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-30T13:07:09.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:09 smithi144 bash[14758]: cluster 2024-06-30T13:07:08.344890+0000 mgr.y (mgr.14144) 128 : cluster [DBG] pgmap v113: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:09.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:09 smithi144 bash[19242]: cluster 2024-06-30T13:07:08.344890+0000 mgr.y (mgr.14144) 128 : cluster [DBG] pgmap v113: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:09.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:09 smithi167 bash[14619]: cluster 2024-06-30T13:07:08.344890+0000 mgr.y (mgr.14144) 128 : cluster [DBG] pgmap v113: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:09.865 INFO:journalctl@ceph.osd.4.smithi167.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:07:10.526 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:10 smithi167 bash[17612]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-30T13:07:10.526 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:10 smithi167 bash[17612]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_4 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2024-06-30T13:07:10.526 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:10 smithi167 bash[17612]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4/block 2024-06-30T13:07:10.527 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:10 smithi167 bash[17612]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-4/block 2024-06-30T13:07:10.527 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:10 smithi167 bash[17612]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-06-30T13:07:10.527 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:10 smithi167 bash[17612]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-06-30T13:07:10.527 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:10 smithi167 bash[17612]: --> ceph-volume lvm activate successful for osd ID: 4 2024-06-30T13:07:11.725 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:11 smithi167 bash[14619]: cluster 2024-06-30T13:07:10.345484+0000 mgr.y (mgr.14144) 129 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:11.725 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:11 smithi167 bash[14619]: audit 2024-06-30T13:07:11.129752+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:11.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:11 smithi144 bash[14758]: cluster 2024-06-30T13:07:10.345484+0000 mgr.y (mgr.14144) 129 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:11.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:11 smithi144 bash[14758]: audit 2024-06-30T13:07:11.129752+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:11.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:11 smithi144 bash[19242]: cluster 2024-06-30T13:07:10.345484+0000 mgr.y (mgr.14144) 129 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:11.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:11 smithi144 bash[19242]: audit 2024-06-30T13:07:11.129752+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:12.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:12 smithi144 bash[14758]: cluster 2024-06-30T13:07:12.346033+0000 mgr.y (mgr.14144) 130 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:12.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:12 smithi144 bash[19242]: cluster 2024-06-30T13:07:12.346033+0000 mgr.y (mgr.14144) 130 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:12.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:12 smithi167 bash[14619]: cluster 2024-06-30T13:07:12.346033+0000 mgr.y (mgr.14144) 130 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:14.026 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:13 smithi167 bash[17918]: debug 2024-06-30T13:07:13.831+0000 7fa106743700 1 -- 172.21.15.167:0/1201928940 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b7c3e9a4e0 con 0x55b7c3f12c00 2024-06-30T13:07:14.590 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-30T13:07:14.591 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-30T13:07:14.591 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-30T13:07:14.591 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-30T13:07:14.591 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0171405 s, 612 MB/s 2024-06-30T13:07:14.591 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-30T13:07:14.637 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:14 smithi167 bash[17918]: debug 2024-06-30T13:07:14.379+0000 7fa10e1ce080 -1 Falling back to public interface 2024-06-30T13:07:15.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:15 smithi167 bash[14619]: cluster 2024-06-30T13:07:14.346579+0000 mgr.y (mgr.14144) 131 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:15.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:15 smithi144 bash[14758]: cluster 2024-06-30T13:07:14.346579+0000 mgr.y (mgr.14144) 131 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:15.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:15 smithi144 bash[19242]: cluster 2024-06-30T13:07:14.346579+0000 mgr.y (mgr.14144) 131 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:15.972 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-06-30T13:07:17.467 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:17 smithi167 bash[17918]: debug 2024-06-30T13:07:17.362+0000 7fa10e1ce080 -1 osd.4 0 log_to_monitors {default=true} 2024-06-30T13:07:17.468 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:17 smithi167 bash[14619]: cluster 2024-06-30T13:07:16.347188+0000 mgr.y (mgr.14144) 132 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:17.468 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:17 smithi167 bash[14619]: audit 2024-06-30T13:07:17.372445+0000 mon.a (mon.0) 339 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-30T13:07:17.468 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:17 smithi167 bash[14619]: audit 2024-06-30T13:07:17.372702+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 [v2:172.21.15.167:6800/828413832,v1:172.21.15.167:6801/828413832]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-30T13:07:17.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:17 smithi144 bash[14758]: cluster 2024-06-30T13:07:16.347188+0000 mgr.y (mgr.14144) 132 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:17.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:17 smithi144 bash[14758]: audit 2024-06-30T13:07:17.372445+0000 mon.a (mon.0) 339 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-30T13:07:17.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:17 smithi144 bash[14758]: audit 2024-06-30T13:07:17.372702+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 [v2:172.21.15.167:6800/828413832,v1:172.21.15.167:6801/828413832]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-30T13:07:17.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:17 smithi144 bash[19242]: cluster 2024-06-30T13:07:16.347188+0000 mgr.y (mgr.14144) 132 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:17.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:17 smithi144 bash[19242]: audit 2024-06-30T13:07:17.372445+0000 mon.a (mon.0) 339 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-30T13:07:17.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:17 smithi144 bash[19242]: audit 2024-06-30T13:07:17.372702+0000 mon.b (mon.1) 9 : audit [INF] from='osd.4 [v2:172.21.15.167:6800/828413832,v1:172.21.15.167:6801/828413832]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-30T13:07:18.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[14758]: audit 2024-06-30T13:07:17.424671+0000 mon.a (mon.0) 340 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-30T13:07:18.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[14758]: cluster 2024-06-30T13:07:17.424857+0000 mon.a (mon.0) 341 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-06-30T13:07:18.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[14758]: audit 2024-06-30T13:07:17.425122+0000 mon.a (mon.0) 342 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:18.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[14758]: audit 2024-06-30T13:07:17.426711+0000 mon.a (mon.0) 343 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:18.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[14758]: audit 2024-06-30T13:07:17.427142+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 [v2:172.21.15.167:6800/828413832,v1:172.21.15.167:6801/828413832]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:18.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[19242]: audit 2024-06-30T13:07:17.424671+0000 mon.a (mon.0) 340 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-30T13:07:18.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[19242]: cluster 2024-06-30T13:07:17.424857+0000 mon.a (mon.0) 341 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-06-30T13:07:18.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[19242]: audit 2024-06-30T13:07:17.425122+0000 mon.a (mon.0) 342 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:18.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[19242]: audit 2024-06-30T13:07:17.426711+0000 mon.a (mon.0) 343 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:18.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:18 smithi144 bash[19242]: audit 2024-06-30T13:07:17.427142+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 [v2:172.21.15.167:6800/828413832,v1:172.21.15.167:6801/828413832]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:18.776 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:07:18 smithi167 bash[17918]: debug 2024-06-30T13:07:18.434+0000 7fa10473f700 -1 osd.4 0 waiting for initial osdmap 2024-06-30T13:07:18.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:18 smithi167 bash[14619]: audit 2024-06-30T13:07:17.424671+0000 mon.a (mon.0) 340 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-30T13:07:18.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:18 smithi167 bash[14619]: cluster 2024-06-30T13:07:17.424857+0000 mon.a (mon.0) 341 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-06-30T13:07:18.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:18 smithi167 bash[14619]: audit 2024-06-30T13:07:17.425122+0000 mon.a (mon.0) 342 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:18.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:18 smithi167 bash[14619]: audit 2024-06-30T13:07:17.426711+0000 mon.a (mon.0) 343 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:18.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:18 smithi167 bash[14619]: audit 2024-06-30T13:07:17.427142+0000 mon.b (mon.1) 10 : audit [INF] from='osd.4 [v2:172.21.15.167:6800/828413832,v1:172.21.15.167:6801/828413832]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:19.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: cluster 2024-06-30T13:07:18.347781+0000 mgr.y (mgr.14144) 133 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: audit 2024-06-30T13:07:18.427795+0000 mon.a (mon.0) 344 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: cluster 2024-06-30T13:07:18.428045+0000 mon.a (mon.0) 345 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: audit 2024-06-30T13:07:18.429321+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: audit 2024-06-30T13:07:18.437049+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: cephadm 2024-06-30T13:07:18.882794+0000 mgr.y (mgr.14144) 134 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: audit 2024-06-30T13:07:18.891541+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: audit 2024-06-30T13:07:18.898984+0000 mon.a (mon.0) 349 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: audit 2024-06-30T13:07:18.902963+0000 mon.a (mon.0) 350 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: audit 2024-06-30T13:07:18.910722+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: cluster 2024-06-30T13:07:19.388582+0000 mon.a (mon.0) 352 : cluster [INF] osd.4 [v2:172.21.15.167:6800/828413832,v1:172.21.15.167:6801/828413832] boot 2024-06-30T13:07:19.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: cluster 2024-06-30T13:07:19.388941+0000 mon.a (mon.0) 353 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-06-30T13:07:19.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:19 smithi167 bash[14619]: audit 2024-06-30T13:07:19.389937+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:19.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: cluster 2024-06-30T13:07:18.347781+0000 mgr.y (mgr.14144) 133 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:19.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: audit 2024-06-30T13:07:18.427795+0000 mon.a (mon.0) 344 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:07:19.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: cluster 2024-06-30T13:07:18.428045+0000 mon.a (mon.0) 345 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-06-30T13:07:19.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: audit 2024-06-30T13:07:18.429321+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:19.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: audit 2024-06-30T13:07:18.437049+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:19.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: cephadm 2024-06-30T13:07:18.882794+0000 mgr.y (mgr.14144) 134 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: audit 2024-06-30T13:07:18.891541+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: audit 2024-06-30T13:07:18.898984+0000 mon.a (mon.0) 349 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: audit 2024-06-30T13:07:18.902963+0000 mon.a (mon.0) 350 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: audit 2024-06-30T13:07:18.910722+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: cluster 2024-06-30T13:07:19.388582+0000 mon.a (mon.0) 352 : cluster [INF] osd.4 [v2:172.21.15.167:6800/828413832,v1:172.21.15.167:6801/828413832] boot 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: cluster 2024-06-30T13:07:19.388941+0000 mon.a (mon.0) 353 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[14758]: audit 2024-06-30T13:07:19.389937+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: cluster 2024-06-30T13:07:18.347781+0000 mgr.y (mgr.14144) 133 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: audit 2024-06-30T13:07:18.427795+0000 mon.a (mon.0) 344 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: cluster 2024-06-30T13:07:18.428045+0000 mon.a (mon.0) 345 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: audit 2024-06-30T13:07:18.429321+0000 mon.a (mon.0) 346 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: audit 2024-06-30T13:07:18.437049+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:19.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: cephadm 2024-06-30T13:07:18.882794+0000 mgr.y (mgr.14144) 134 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:07:19.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: audit 2024-06-30T13:07:18.891541+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:19.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: audit 2024-06-30T13:07:18.898984+0000 mon.a (mon.0) 349 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:19.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: audit 2024-06-30T13:07:18.902963+0000 mon.a (mon.0) 350 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:19.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: audit 2024-06-30T13:07:18.910722+0000 mon.a (mon.0) 351 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:19.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: cluster 2024-06-30T13:07:19.388582+0000 mon.a (mon.0) 352 : cluster [INF] osd.4 [v2:172.21.15.167:6800/828413832,v1:172.21.15.167:6801/828413832] boot 2024-06-30T13:07:19.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: cluster 2024-06-30T13:07:19.388941+0000 mon.a (mon.0) 353 : cluster [DBG] osdmap e30: 5 total, 5 up, 5 in 2024-06-30T13:07:19.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:19 smithi144 bash[19242]: audit 2024-06-30T13:07:19.389937+0000 mon.a (mon.0) 354 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-30T13:07:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[14758]: cluster 2024-06-30T13:07:18.366256+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:07:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[14758]: cluster 2024-06-30T13:07:18.366447+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:07:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[14758]: audit 2024-06-30T13:07:19.657235+0000 mgr.y (mgr.14144) 135 : audit [DBG] from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:07:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[14758]: audit 2024-06-30T13:07:19.658916+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:07:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[14758]: cephadm 2024-06-30T13:07:19.662228+0000 mgr.y (mgr.14144) 136 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:07:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[14758]: audit 2024-06-30T13:07:19.662866+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:07:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[14758]: audit 2024-06-30T13:07:19.664947+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[14758]: cluster 2024-06-30T13:07:20.348361+0000 mgr.y (mgr.14144) 137 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[14758]: cluster 2024-06-30T13:07:20.391761+0000 mon.a (mon.0) 358 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-06-30T13:07:20.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[19242]: cluster 2024-06-30T13:07:18.366256+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:07:20.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[19242]: cluster 2024-06-30T13:07:18.366447+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:07:20.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[19242]: audit 2024-06-30T13:07:19.657235+0000 mgr.y (mgr.14144) 135 : audit [DBG] from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:07:20.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[19242]: audit 2024-06-30T13:07:19.658916+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:07:20.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[19242]: cephadm 2024-06-30T13:07:19.662228+0000 mgr.y (mgr.14144) 136 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:07:20.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[19242]: audit 2024-06-30T13:07:19.662866+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:07:20.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[19242]: audit 2024-06-30T13:07:19.664947+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:20.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[19242]: cluster 2024-06-30T13:07:20.348361+0000 mgr.y (mgr.14144) 137 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:20.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:20 smithi144 bash[19242]: cluster 2024-06-30T13:07:20.391761+0000 mon.a (mon.0) 358 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-06-30T13:07:20.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:20 smithi167 bash[14619]: cluster 2024-06-30T13:07:18.366256+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:07:20.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:20 smithi167 bash[14619]: cluster 2024-06-30T13:07:18.366447+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:07:20.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:20 smithi167 bash[14619]: audit 2024-06-30T13:07:19.657235+0000 mgr.y (mgr.14144) 135 : audit [DBG] from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:07:20.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:20 smithi167 bash[14619]: audit 2024-06-30T13:07:19.658916+0000 mon.a (mon.0) 355 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:07:20.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:20 smithi167 bash[14619]: cephadm 2024-06-30T13:07:19.662228+0000 mgr.y (mgr.14144) 136 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:07:20.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:20 smithi167 bash[14619]: audit 2024-06-30T13:07:19.662866+0000 mon.a (mon.0) 356 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:07:20.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:20 smithi167 bash[14619]: audit 2024-06-30T13:07:19.664947+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:20.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:20 smithi167 bash[14619]: cluster 2024-06-30T13:07:20.348361+0000 mgr.y (mgr.14144) 137 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:20.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:20 smithi167 bash[14619]: cluster 2024-06-30T13:07:20.391761+0000 mon.a (mon.0) 358 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-06-30T13:07:22.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:22 smithi144 bash[14758]: cluster 2024-06-30T13:07:21.391760+0000 mon.a (mon.0) 359 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-06-30T13:07:22.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:22 smithi144 bash[19242]: cluster 2024-06-30T13:07:21.391760+0000 mon.a (mon.0) 359 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-06-30T13:07:22.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:22 smithi167 bash[14619]: cluster 2024-06-30T13:07:21.391760+0000 mon.a (mon.0) 359 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-06-30T13:07:23.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:23 smithi144 bash[14758]: cluster 2024-06-30T13:07:22.348952+0000 mgr.y (mgr.14144) 138 : cluster [DBG] pgmap v125: 1 pgs: 1 peering; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:23.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:23 smithi144 bash[19242]: cluster 2024-06-30T13:07:22.348952+0000 mgr.y (mgr.14144) 138 : cluster [DBG] pgmap v125: 1 pgs: 1 peering; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:23.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:23 smithi167 bash[14619]: cluster 2024-06-30T13:07:22.348952+0000 mgr.y (mgr.14144) 138 : cluster [DBG] pgmap v125: 1 pgs: 1 peering; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:24.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[14758]: audit 2024-06-30T13:07:23.449272+0000 mon.a (mon.0) 360 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "107635de-2830-4c26-915a-04791174406d"}]: dispatch 2024-06-30T13:07:24.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[14758]: audit 2024-06-30T13:07:23.449393+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.167:0/3581047150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "107635de-2830-4c26-915a-04791174406d"}]: dispatch 2024-06-30T13:07:24.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[14758]: audit 2024-06-30T13:07:23.459642+0000 mon.a (mon.0) 361 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "107635de-2830-4c26-915a-04791174406d"}]': finished 2024-06-30T13:07:24.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[14758]: cluster 2024-06-30T13:07:23.459808+0000 mon.a (mon.0) 362 : cluster [DBG] osdmap e33: 6 total, 5 up, 6 in 2024-06-30T13:07:24.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[14758]: audit 2024-06-30T13:07:23.460102+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:24.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[14758]: audit 2024-06-30T13:07:24.128347+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.167:0/3975723861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:07:24.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[19242]: audit 2024-06-30T13:07:23.449272+0000 mon.a (mon.0) 360 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "107635de-2830-4c26-915a-04791174406d"}]: dispatch 2024-06-30T13:07:24.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[19242]: audit 2024-06-30T13:07:23.449393+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.167:0/3581047150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "107635de-2830-4c26-915a-04791174406d"}]: dispatch 2024-06-30T13:07:24.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[19242]: audit 2024-06-30T13:07:23.459642+0000 mon.a (mon.0) 361 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "107635de-2830-4c26-915a-04791174406d"}]': finished 2024-06-30T13:07:24.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[19242]: cluster 2024-06-30T13:07:23.459808+0000 mon.a (mon.0) 362 : cluster [DBG] osdmap e33: 6 total, 5 up, 6 in 2024-06-30T13:07:24.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[19242]: audit 2024-06-30T13:07:23.460102+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:24.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:24 smithi144 bash[19242]: audit 2024-06-30T13:07:24.128347+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.167:0/3975723861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:07:24.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:24 smithi167 bash[14619]: audit 2024-06-30T13:07:23.449272+0000 mon.a (mon.0) 360 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "107635de-2830-4c26-915a-04791174406d"}]: dispatch 2024-06-30T13:07:24.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:24 smithi167 bash[14619]: audit 2024-06-30T13:07:23.449393+0000 mon.b (mon.1) 11 : audit [INF] from='client.? 172.21.15.167:0/3581047150' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "107635de-2830-4c26-915a-04791174406d"}]: dispatch 2024-06-30T13:07:24.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:24 smithi167 bash[14619]: audit 2024-06-30T13:07:23.459642+0000 mon.a (mon.0) 361 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "107635de-2830-4c26-915a-04791174406d"}]': finished 2024-06-30T13:07:24.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:24 smithi167 bash[14619]: cluster 2024-06-30T13:07:23.459808+0000 mon.a (mon.0) 362 : cluster [DBG] osdmap e33: 6 total, 5 up, 6 in 2024-06-30T13:07:24.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:24 smithi167 bash[14619]: audit 2024-06-30T13:07:23.460102+0000 mon.a (mon.0) 363 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:24.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:24 smithi167 bash[14619]: audit 2024-06-30T13:07:24.128347+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? 172.21.15.167:0/3975723861' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:07:25.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:25 smithi144 bash[19242]: cluster 2024-06-30T13:07:24.349583+0000 mgr.y (mgr.14144) 139 : cluster [DBG] pgmap v127: 1 pgs: 1 peering; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:25.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:25 smithi144 bash[14758]: cluster 2024-06-30T13:07:24.349583+0000 mgr.y (mgr.14144) 139 : cluster [DBG] pgmap v127: 1 pgs: 1 peering; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:25.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:25 smithi167 bash[14619]: cluster 2024-06-30T13:07:24.349583+0000 mgr.y (mgr.14144) 139 : cluster [DBG] pgmap v127: 1 pgs: 1 peering; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:27.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:27 smithi144 bash[19242]: cluster 2024-06-30T13:07:26.350174+0000 mgr.y (mgr.14144) 140 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:27.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:27 smithi144 bash[14758]: cluster 2024-06-30T13:07:26.350174+0000 mgr.y (mgr.14144) 140 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:27.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:27 smithi167 bash[14619]: cluster 2024-06-30T13:07:26.350174+0000 mgr.y (mgr.14144) 140 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:29.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:29 smithi167 bash[14619]: cluster 2024-06-30T13:07:28.350776+0000 mgr.y (mgr.14144) 141 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:29.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:29 smithi144 bash[14758]: cluster 2024-06-30T13:07:28.350776+0000 mgr.y (mgr.14144) 141 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:29.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:29 smithi144 bash[19242]: cluster 2024-06-30T13:07:28.350776+0000 mgr.y (mgr.14144) 141 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:31.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:31 smithi144 bash[19242]: cluster 2024-06-30T13:07:30.351370+0000 mgr.y (mgr.14144) 142 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:31.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:31 smithi144 bash[19242]: audit 2024-06-30T13:07:31.293167+0000 mon.a (mon.0) 364 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-30T13:07:31.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:31 smithi144 bash[19242]: audit 2024-06-30T13:07:31.295006+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:31.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:31 smithi144 bash[14758]: cluster 2024-06-30T13:07:30.351370+0000 mgr.y (mgr.14144) 142 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:31.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:31 smithi144 bash[14758]: audit 2024-06-30T13:07:31.293167+0000 mon.a (mon.0) 364 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-30T13:07:31.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:31 smithi144 bash[14758]: audit 2024-06-30T13:07:31.295006+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:31.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:31 smithi167 bash[14619]: cluster 2024-06-30T13:07:30.351370+0000 mgr.y (mgr.14144) 142 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:31.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:31 smithi167 bash[14619]: audit 2024-06-30T13:07:31.293167+0000 mon.a (mon.0) 364 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-30T13:07:31.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:31 smithi167 bash[14619]: audit 2024-06-30T13:07:31.295006+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:32.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:32 smithi144 bash[19242]: cephadm 2024-06-30T13:07:31.296483+0000 mgr.y (mgr.14144) 143 : cephadm [INF] Deploying daemon osd.5 on smithi167 2024-06-30T13:07:32.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:32 smithi144 bash[14758]: cephadm 2024-06-30T13:07:31.296483+0000 mgr.y (mgr.14144) 143 : cephadm [INF] Deploying daemon osd.5 on smithi167 2024-06-30T13:07:32.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:32 smithi167 bash[14619]: cephadm 2024-06-30T13:07:31.296483+0000 mgr.y (mgr.14144) 143 : cephadm [INF] Deploying daemon osd.5 on smithi167 2024-06-30T13:07:33.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:33 smithi144 bash[14758]: cluster 2024-06-30T13:07:32.351943+0000 mgr.y (mgr.14144) 144 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:33.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:33 smithi144 bash[19242]: cluster 2024-06-30T13:07:32.351943+0000 mgr.y (mgr.14144) 144 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:33.774 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:33 smithi167 bash[14619]: cluster 2024-06-30T13:07:32.351943+0000 mgr.y (mgr.14144) 144 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:33.827 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-06-30T13:07:34.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:34 smithi144 bash[14758]: audit 2024-06-30T13:07:33.806891+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:34.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:34 smithi144 bash[14758]: audit 2024-06-30T13:07:33.815044+0000 mon.a (mon.0) 367 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:34.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:34 smithi144 bash[14758]: audit 2024-06-30T13:07:33.825446+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:34.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:34 smithi144 bash[19242]: audit 2024-06-30T13:07:33.806891+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:34.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:34 smithi144 bash[19242]: audit 2024-06-30T13:07:33.815044+0000 mon.a (mon.0) 367 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:34.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:34 smithi144 bash[19242]: audit 2024-06-30T13:07:33.825446+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:34.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:34 smithi167 bash[14619]: audit 2024-06-30T13:07:33.806891+0000 mon.a (mon.0) 366 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:34.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:34 smithi167 bash[14619]: audit 2024-06-30T13:07:33.815044+0000 mon.a (mon.0) 367 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:34.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:34 smithi167 bash[14619]: audit 2024-06-30T13:07:33.825446+0000 mon.a (mon.0) 368 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:35.170 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.5.service 2024-06-30T13:07:35.173 INFO:tasks.cephadm:Deploying osd.6 on smithi167 with /dev/vg_nvme/lv_2... 2024-06-30T13:07:35.173 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-30T13:07:35.683 INFO:journalctl@ceph.osd.5.smithi167.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:07:35.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:35 smithi144 bash[14758]: cluster 2024-06-30T13:07:34.352571+0000 mgr.y (mgr.14144) 145 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:35.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:35 smithi144 bash[19242]: cluster 2024-06-30T13:07:34.352571+0000 mgr.y (mgr.14144) 145 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:35.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:35 smithi167 bash[14619]: cluster 2024-06-30T13:07:34.352571+0000 mgr.y (mgr.14144) 145 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:36.276 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:35 smithi167 bash[20296]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-30T13:07:36.276 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:35 smithi167 bash[20296]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_3 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2024-06-30T13:07:36.277 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:35 smithi167 bash[20296]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5/block 2024-06-30T13:07:36.277 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:35 smithi167 bash[20296]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-5/block 2024-06-30T13:07:36.277 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:35 smithi167 bash[20296]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-06-30T13:07:36.277 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:35 smithi167 bash[20296]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-06-30T13:07:36.277 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:35 smithi167 bash[20296]: --> ceph-volume lvm activate successful for osd ID: 5 2024-06-30T13:07:36.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:36 smithi144 bash[19242]: cluster 2024-06-30T13:07:36.353127+0000 mgr.y (mgr.14144) 146 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:36.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:36 smithi144 bash[14758]: cluster 2024-06-30T13:07:36.353127+0000 mgr.y (mgr.14144) 146 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:36.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:36 smithi167 bash[14619]: cluster 2024-06-30T13:07:36.353127+0000 mgr.y (mgr.14144) 146 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:39.250 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:38 smithi167 bash[14619]: audit 2024-06-30T13:07:37.853618+0000 mon.a (mon.0) 369 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:39.250 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:38 smithi167 bash[14619]: cluster 2024-06-30T13:07:38.353691+0000 mgr.y (mgr.14144) 147 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:39.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:38 smithi144 bash[14758]: audit 2024-06-30T13:07:37.853618+0000 mon.a (mon.0) 369 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:39.258 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:38 smithi144 bash[14758]: cluster 2024-06-30T13:07:38.353691+0000 mgr.y (mgr.14144) 147 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:39.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:38 smithi144 bash[19242]: audit 2024-06-30T13:07:37.853618+0000 mon.a (mon.0) 369 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:39.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:38 smithi144 bash[19242]: cluster 2024-06-30T13:07:38.353691+0000 mgr.y (mgr.14144) 147 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:39.526 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:39 smithi167 bash[20455]: debug 2024-06-30T13:07:39.402+0000 7f940e289700 1 -- 172.21.15.167:0/3322961058 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5636421744e0 con 0x5636421ecc00 2024-06-30T13:07:40.276 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:39 smithi167 bash[20455]: debug 2024-06-30T13:07:39.942+0000 7f9415d14080 -1 Falling back to public interface 2024-06-30T13:07:41.009 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-30T13:07:41.010 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-30T13:07:41.010 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-30T13:07:41.010 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-30T13:07:41.010 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0177988 s, 589 MB/s 2024-06-30T13:07:41.010 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-30T13:07:41.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:41 smithi167 bash[14619]: cluster 2024-06-30T13:07:40.354244+0000 mgr.y (mgr.14144) 148 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:41.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:41 smithi144 bash[14758]: cluster 2024-06-30T13:07:40.354244+0000 mgr.y (mgr.14144) 148 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:41.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:41 smithi144 bash[19242]: cluster 2024-06-30T13:07:40.354244+0000 mgr.y (mgr.14144) 148 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:42.264 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-06-30T13:07:43.276 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:43 smithi167 bash[20455]: debug 2024-06-30T13:07:42.998+0000 7f9415d14080 -1 osd.5 0 log_to_monitors {default=true} 2024-06-30T13:07:43.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:43 smithi167 bash[14619]: cluster 2024-06-30T13:07:42.354815+0000 mgr.y (mgr.14144) 149 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:43.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:43 smithi167 bash[14619]: audit 2024-06-30T13:07:43.004958+0000 mon.a (mon.0) 370 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-30T13:07:43.664 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:43 smithi167 bash[14619]: audit 2024-06-30T13:07:43.005337+0000 mon.b (mon.1) 13 : audit [INF] from='osd.5 [v2:172.21.15.167:6808/3971163726,v1:172.21.15.167:6809/3971163726]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-30T13:07:43.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:43 smithi144 bash[14758]: cluster 2024-06-30T13:07:42.354815+0000 mgr.y (mgr.14144) 149 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:43.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:43 smithi144 bash[14758]: audit 2024-06-30T13:07:43.004958+0000 mon.a (mon.0) 370 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-30T13:07:43.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:43 smithi144 bash[14758]: audit 2024-06-30T13:07:43.005337+0000 mon.b (mon.1) 13 : audit [INF] from='osd.5 [v2:172.21.15.167:6808/3971163726,v1:172.21.15.167:6809/3971163726]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-30T13:07:43.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:43 smithi144 bash[19242]: cluster 2024-06-30T13:07:42.354815+0000 mgr.y (mgr.14144) 149 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:43.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:43 smithi144 bash[19242]: audit 2024-06-30T13:07:43.004958+0000 mon.a (mon.0) 370 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-30T13:07:43.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:43 smithi144 bash[19242]: audit 2024-06-30T13:07:43.005337+0000 mon.b (mon.1) 13 : audit [INF] from='osd.5 [v2:172.21.15.167:6808/3971163726,v1:172.21.15.167:6809/3971163726]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-30T13:07:44.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[14758]: audit 2024-06-30T13:07:43.402964+0000 mon.a (mon.0) 371 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-30T13:07:44.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[14758]: cluster 2024-06-30T13:07:43.403119+0000 mon.a (mon.0) 372 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-06-30T13:07:44.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[14758]: audit 2024-06-30T13:07:43.403975+0000 mon.a (mon.0) 373 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:44.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[14758]: audit 2024-06-30T13:07:43.405933+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 [v2:172.21.15.167:6808/3971163726,v1:172.21.15.167:6809/3971163726]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:44.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[14758]: audit 2024-06-30T13:07:43.410723+0000 mon.a (mon.0) 374 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:44.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[19242]: audit 2024-06-30T13:07:43.402964+0000 mon.a (mon.0) 371 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-30T13:07:44.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[19242]: cluster 2024-06-30T13:07:43.403119+0000 mon.a (mon.0) 372 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-06-30T13:07:44.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[19242]: audit 2024-06-30T13:07:43.403975+0000 mon.a (mon.0) 373 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:44.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[19242]: audit 2024-06-30T13:07:43.405933+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 [v2:172.21.15.167:6808/3971163726,v1:172.21.15.167:6809/3971163726]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:44.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:44 smithi144 bash[19242]: audit 2024-06-30T13:07:43.410723+0000 mon.a (mon.0) 374 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:44.776 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:07:44 smithi167 bash[20455]: debug 2024-06-30T13:07:44.426+0000 7f940c285700 -1 osd.5 0 waiting for initial osdmap 2024-06-30T13:07:44.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:44 smithi167 bash[14619]: audit 2024-06-30T13:07:43.402964+0000 mon.a (mon.0) 371 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-30T13:07:44.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:44 smithi167 bash[14619]: cluster 2024-06-30T13:07:43.403119+0000 mon.a (mon.0) 372 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-06-30T13:07:44.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:44 smithi167 bash[14619]: audit 2024-06-30T13:07:43.403975+0000 mon.a (mon.0) 373 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:44.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:44 smithi167 bash[14619]: audit 2024-06-30T13:07:43.405933+0000 mon.b (mon.1) 14 : audit [INF] from='osd.5 [v2:172.21.15.167:6808/3971163726,v1:172.21.15.167:6809/3971163726]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:44.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:44 smithi167 bash[14619]: audit 2024-06-30T13:07:43.410723+0000 mon.a (mon.0) 374 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:07:45.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:45 smithi167 bash[14619]: cluster 2024-06-30T13:07:44.355386+0000 mgr.y (mgr.14144) 150 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:45.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:45 smithi167 bash[14619]: audit 2024-06-30T13:07:44.413454+0000 mon.a (mon.0) 375 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:07:45.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:45 smithi167 bash[14619]: cluster 2024-06-30T13:07:44.413677+0000 mon.a (mon.0) 376 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-06-30T13:07:45.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:45 smithi167 bash[14619]: audit 2024-06-30T13:07:44.414541+0000 mon.a (mon.0) 377 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:45.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:45 smithi167 bash[14619]: audit 2024-06-30T13:07:44.426247+0000 mon.a (mon.0) 378 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:45.718 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:45 smithi167 bash[14619]: audit 2024-06-30T13:07:45.310260+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:45.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[14758]: cluster 2024-06-30T13:07:44.355386+0000 mgr.y (mgr.14144) 150 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[14758]: audit 2024-06-30T13:07:44.413454+0000 mon.a (mon.0) 375 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:07:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[14758]: cluster 2024-06-30T13:07:44.413677+0000 mon.a (mon.0) 376 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-06-30T13:07:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[14758]: audit 2024-06-30T13:07:44.414541+0000 mon.a (mon.0) 377 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[14758]: audit 2024-06-30T13:07:44.426247+0000 mon.a (mon.0) 378 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[14758]: audit 2024-06-30T13:07:45.310260+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:45.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[19242]: cluster 2024-06-30T13:07:44.355386+0000 mgr.y (mgr.14144) 150 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:45.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[19242]: audit 2024-06-30T13:07:44.413454+0000 mon.a (mon.0) 375 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:07:45.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[19242]: cluster 2024-06-30T13:07:44.413677+0000 mon.a (mon.0) 376 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-06-30T13:07:45.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[19242]: audit 2024-06-30T13:07:44.414541+0000 mon.a (mon.0) 377 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:45.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[19242]: audit 2024-06-30T13:07:44.426247+0000 mon.a (mon.0) 378 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:45.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:45 smithi144 bash[19242]: audit 2024-06-30T13:07:45.310260+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: cluster 2024-06-30T13:07:43.979362+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:07:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: cluster 2024-06-30T13:07:43.979569+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:07:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: cephadm 2024-06-30T13:07:45.300741+0000 mgr.y (mgr.14144) 151 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:07:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: audit 2024-06-30T13:07:45.427535+0000 mon.a (mon.0) 380 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: cluster 2024-06-30T13:07:45.433448+0000 mon.a (mon.0) 381 : cluster [INF] osd.5 [v2:172.21.15.167:6808/3971163726,v1:172.21.15.167:6809/3971163726] boot 2024-06-30T13:07:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: cluster 2024-06-30T13:07:45.433552+0000 mon.a (mon.0) 382 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2024-06-30T13:07:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: audit 2024-06-30T13:07:45.433812+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: audit 2024-06-30T13:07:45.558064+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:46.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: audit 2024-06-30T13:07:45.561706+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: audit 2024-06-30T13:07:45.571879+0000 mon.a (mon.0) 386 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: audit 2024-06-30T13:07:46.065854+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: audit 2024-06-30T13:07:46.069421+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[14758]: audit 2024-06-30T13:07:46.071260+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: cluster 2024-06-30T13:07:43.979362+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: cluster 2024-06-30T13:07:43.979569+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: cephadm 2024-06-30T13:07:45.300741+0000 mgr.y (mgr.14144) 151 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: audit 2024-06-30T13:07:45.427535+0000 mon.a (mon.0) 380 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: cluster 2024-06-30T13:07:45.433448+0000 mon.a (mon.0) 381 : cluster [INF] osd.5 [v2:172.21.15.167:6808/3971163726,v1:172.21.15.167:6809/3971163726] boot 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: cluster 2024-06-30T13:07:45.433552+0000 mon.a (mon.0) 382 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: audit 2024-06-30T13:07:45.433812+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: audit 2024-06-30T13:07:45.558064+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: audit 2024-06-30T13:07:45.561706+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: audit 2024-06-30T13:07:45.571879+0000 mon.a (mon.0) 386 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:46.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: audit 2024-06-30T13:07:46.065854+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:07:46.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: audit 2024-06-30T13:07:46.069421+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:07:46.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:46 smithi144 bash[19242]: audit 2024-06-30T13:07:46.071260+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:46.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: cluster 2024-06-30T13:07:43.979362+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:07:46.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: cluster 2024-06-30T13:07:43.979569+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:07:46.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: cephadm 2024-06-30T13:07:45.300741+0000 mgr.y (mgr.14144) 151 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: audit 2024-06-30T13:07:45.427535+0000 mon.a (mon.0) 380 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: cluster 2024-06-30T13:07:45.433448+0000 mon.a (mon.0) 381 : cluster [INF] osd.5 [v2:172.21.15.167:6808/3971163726,v1:172.21.15.167:6809/3971163726] boot 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: cluster 2024-06-30T13:07:45.433552+0000 mon.a (mon.0) 382 : cluster [DBG] osdmap e36: 6 total, 6 up, 6 in 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: audit 2024-06-30T13:07:45.433812+0000 mon.a (mon.0) 383 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: audit 2024-06-30T13:07:45.558064+0000 mon.a (mon.0) 384 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: audit 2024-06-30T13:07:45.561706+0000 mon.a (mon.0) 385 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: audit 2024-06-30T13:07:45.571879+0000 mon.a (mon.0) 386 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: audit 2024-06-30T13:07:46.065854+0000 mon.a (mon.0) 387 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: audit 2024-06-30T13:07:46.069421+0000 mon.a (mon.0) 388 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:07:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:46 smithi167 bash[14619]: audit 2024-06-30T13:07:46.071260+0000 mon.a (mon.0) 389 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:47.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:47 smithi167 bash[14619]: audit 2024-06-30T13:07:46.064249+0000 mgr.y (mgr.14144) 152 : audit [DBG] from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:07:47.646 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:47 smithi167 bash[14619]: cephadm 2024-06-30T13:07:46.068860+0000 mgr.y (mgr.14144) 153 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:07:47.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:47 smithi167 bash[14619]: cluster 2024-06-30T13:07:46.355959+0000 mgr.y (mgr.14144) 154 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:47.647 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:47 smithi167 bash[14619]: cluster 2024-06-30T13:07:46.436303+0000 mon.a (mon.0) 390 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-06-30T13:07:47.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:47 smithi144 bash[14758]: audit 2024-06-30T13:07:46.064249+0000 mgr.y (mgr.14144) 152 : audit [DBG] from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:07:47.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:47 smithi144 bash[14758]: cephadm 2024-06-30T13:07:46.068860+0000 mgr.y (mgr.14144) 153 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:07:47.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:47 smithi144 bash[14758]: cluster 2024-06-30T13:07:46.355959+0000 mgr.y (mgr.14144) 154 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:47.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:47 smithi144 bash[14758]: cluster 2024-06-30T13:07:46.436303+0000 mon.a (mon.0) 390 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-06-30T13:07:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:47 smithi144 bash[19242]: audit 2024-06-30T13:07:46.064249+0000 mgr.y (mgr.14144) 152 : audit [DBG] from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:07:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:47 smithi144 bash[19242]: cephadm 2024-06-30T13:07:46.068860+0000 mgr.y (mgr.14144) 153 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:07:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:47 smithi144 bash[19242]: cluster 2024-06-30T13:07:46.355959+0000 mgr.y (mgr.14144) 154 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2024-06-30T13:07:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:47 smithi144 bash[19242]: cluster 2024-06-30T13:07:46.436303+0000 mon.a (mon.0) 390 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-06-30T13:07:48.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:48 smithi167 bash[14619]: cluster 2024-06-30T13:07:47.440661+0000 mon.a (mon.0) 391 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-06-30T13:07:48.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:48 smithi167 bash[14619]: cluster 2024-06-30T13:07:48.356562+0000 mgr.y (mgr.14144) 155 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:48.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:48 smithi144 bash[14758]: cluster 2024-06-30T13:07:47.440661+0000 mon.a (mon.0) 391 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-06-30T13:07:48.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:48 smithi144 bash[14758]: cluster 2024-06-30T13:07:48.356562+0000 mgr.y (mgr.14144) 155 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:48.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:48 smithi144 bash[19242]: cluster 2024-06-30T13:07:47.440661+0000 mon.a (mon.0) 391 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-06-30T13:07:48.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:48 smithi144 bash[19242]: cluster 2024-06-30T13:07:48.356562+0000 mgr.y (mgr.14144) 155 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[14758]: audit 2024-06-30T13:07:50.057354+0000 mon.a (mon.0) 392 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b80a36b1-805a-4794-8b5b-53ffd9bd873c"}]: dispatch 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[14758]: audit 2024-06-30T13:07:50.057684+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.167:0/1118854902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b80a36b1-805a-4794-8b5b-53ffd9bd873c"}]: dispatch 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[14758]: audit 2024-06-30T13:07:50.071204+0000 mon.a (mon.0) 393 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b80a36b1-805a-4794-8b5b-53ffd9bd873c"}]': finished 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[14758]: cluster 2024-06-30T13:07:50.071406+0000 mon.a (mon.0) 394 : cluster [DBG] osdmap e39: 7 total, 6 up, 7 in 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[14758]: audit 2024-06-30T13:07:50.071692+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[19242]: audit 2024-06-30T13:07:50.057354+0000 mon.a (mon.0) 392 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b80a36b1-805a-4794-8b5b-53ffd9bd873c"}]: dispatch 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[19242]: audit 2024-06-30T13:07:50.057684+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.167:0/1118854902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b80a36b1-805a-4794-8b5b-53ffd9bd873c"}]: dispatch 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[19242]: audit 2024-06-30T13:07:50.071204+0000 mon.a (mon.0) 393 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b80a36b1-805a-4794-8b5b-53ffd9bd873c"}]': finished 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[19242]: cluster 2024-06-30T13:07:50.071406+0000 mon.a (mon.0) 394 : cluster [DBG] osdmap e39: 7 total, 6 up, 7 in 2024-06-30T13:07:50.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:50 smithi144 bash[19242]: audit 2024-06-30T13:07:50.071692+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:07:50.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:50 smithi167 bash[14619]: audit 2024-06-30T13:07:50.057354+0000 mon.a (mon.0) 392 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b80a36b1-805a-4794-8b5b-53ffd9bd873c"}]: dispatch 2024-06-30T13:07:50.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:50 smithi167 bash[14619]: audit 2024-06-30T13:07:50.057684+0000 mon.b (mon.1) 15 : audit [INF] from='client.? 172.21.15.167:0/1118854902' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b80a36b1-805a-4794-8b5b-53ffd9bd873c"}]: dispatch 2024-06-30T13:07:50.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:50 smithi167 bash[14619]: audit 2024-06-30T13:07:50.071204+0000 mon.a (mon.0) 393 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b80a36b1-805a-4794-8b5b-53ffd9bd873c"}]': finished 2024-06-30T13:07:50.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:50 smithi167 bash[14619]: cluster 2024-06-30T13:07:50.071406+0000 mon.a (mon.0) 394 : cluster [DBG] osdmap e39: 7 total, 6 up, 7 in 2024-06-30T13:07:50.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:50 smithi167 bash[14619]: audit 2024-06-30T13:07:50.071692+0000 mon.a (mon.0) 395 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:07:51.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:51 smithi144 bash[14758]: cluster 2024-06-30T13:07:50.357124+0000 mgr.y (mgr.14144) 156 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:51.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:51 smithi144 bash[14758]: audit 2024-06-30T13:07:50.724740+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.167:0/2081731733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:07:51.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:51 smithi144 bash[19242]: cluster 2024-06-30T13:07:50.357124+0000 mgr.y (mgr.14144) 156 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:51.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:51 smithi144 bash[19242]: audit 2024-06-30T13:07:50.724740+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.167:0/2081731733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:07:51.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:51 smithi167 bash[14619]: cluster 2024-06-30T13:07:50.357124+0000 mgr.y (mgr.14144) 156 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:51.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:51 smithi167 bash[14619]: audit 2024-06-30T13:07:50.724740+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? 172.21.15.167:0/2081731733' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:07:53.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:53 smithi144 bash[14758]: cluster 2024-06-30T13:07:52.357829+0000 mgr.y (mgr.14144) 157 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:53.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:53 smithi144 bash[19242]: cluster 2024-06-30T13:07:52.357829+0000 mgr.y (mgr.14144) 157 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:53.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:53 smithi167 bash[14619]: cluster 2024-06-30T13:07:52.357829+0000 mgr.y (mgr.14144) 157 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:55.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:55 smithi167 bash[14619]: cluster 2024-06-30T13:07:54.358447+0000 mgr.y (mgr.14144) 158 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:55.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:55 smithi144 bash[14758]: cluster 2024-06-30T13:07:54.358447+0000 mgr.y (mgr.14144) 158 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:55.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:55 smithi144 bash[19242]: cluster 2024-06-30T13:07:54.358447+0000 mgr.y (mgr.14144) 158 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:57.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:57 smithi144 bash[14758]: cluster 2024-06-30T13:07:56.359042+0000 mgr.y (mgr.14144) 159 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:57.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:57 smithi144 bash[14758]: audit 2024-06-30T13:07:56.504662+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:07:57.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:57 smithi144 bash[14758]: audit 2024-06-30T13:07:56.507103+0000 mon.a (mon.0) 397 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:07:57.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:57 smithi144 bash[19242]: cluster 2024-06-30T13:07:56.359042+0000 mgr.y (mgr.14144) 159 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:57.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:57 smithi144 bash[19242]: audit 2024-06-30T13:07:56.504662+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:07:57.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:57 smithi144 bash[19242]: audit 2024-06-30T13:07:56.507103+0000 mon.a (mon.0) 397 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:07:57.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:57 smithi167 bash[14619]: cluster 2024-06-30T13:07:56.359042+0000 mgr.y (mgr.14144) 159 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:57.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:57 smithi167 bash[14619]: audit 2024-06-30T13:07:56.504662+0000 mon.a (mon.0) 396 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:07:57.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:57 smithi167 bash[14619]: audit 2024-06-30T13:07:56.507103+0000 mon.a (mon.0) 397 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:07:58.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:58 smithi167 bash[14619]: audit 2024-06-30T13:07:57.959710+0000 mon.a (mon.0) 398 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-30T13:07:58.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:58 smithi167 bash[14619]: audit 2024-06-30T13:07:57.961507+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:58.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:58 smithi144 bash[14758]: audit 2024-06-30T13:07:57.959710+0000 mon.a (mon.0) 398 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-30T13:07:58.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:58 smithi144 bash[14758]: audit 2024-06-30T13:07:57.961507+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:58.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:58 smithi144 bash[19242]: audit 2024-06-30T13:07:57.959710+0000 mon.a (mon.0) 398 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-30T13:07:58.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:58 smithi144 bash[19242]: audit 2024-06-30T13:07:57.961507+0000 mon.a (mon.0) 399 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:07:59.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:59 smithi144 bash[14758]: cephadm 2024-06-30T13:07:57.963009+0000 mgr.y (mgr.14144) 160 : cephadm [INF] Deploying daemon osd.6 on smithi167 2024-06-30T13:07:59.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:07:59 smithi144 bash[14758]: cluster 2024-06-30T13:07:58.359618+0000 mgr.y (mgr.14144) 161 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:59.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:59 smithi144 bash[19242]: cephadm 2024-06-30T13:07:57.963009+0000 mgr.y (mgr.14144) 160 : cephadm [INF] Deploying daemon osd.6 on smithi167 2024-06-30T13:07:59.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:07:59 smithi144 bash[19242]: cluster 2024-06-30T13:07:58.359618+0000 mgr.y (mgr.14144) 161 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:07:59.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:59 smithi167 bash[14619]: cephadm 2024-06-30T13:07:57.963009+0000 mgr.y (mgr.14144) 160 : cephadm [INF] Deploying daemon osd.6 on smithi167 2024-06-30T13:07:59.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:07:59 smithi167 bash[14619]: cluster 2024-06-30T13:07:58.359618+0000 mgr.y (mgr.14144) 161 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:00.523 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 6 on host 'smithi167' 2024-06-30T13:08:01.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:01 smithi167 bash[14619]: cluster 2024-06-30T13:08:00.360304+0000 mgr.y (mgr.14144) 162 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:01.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:01 smithi167 bash[14619]: audit 2024-06-30T13:08:00.498314+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:01.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:01 smithi167 bash[14619]: audit 2024-06-30T13:08:00.509054+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:01.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:01 smithi167 bash[14619]: audit 2024-06-30T13:08:00.521077+0000 mon.a (mon.0) 402 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:01.718 DEBUG:teuthology.orchestra.run.smithi167:osd.6> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.6.service 2024-06-30T13:08:01.720 INFO:tasks.cephadm:Deploying osd.7 on smithi167 with /dev/vg_nvme/lv_1... 2024-06-30T13:08:01.721 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-30T13:08:01.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:01 smithi144 bash[14758]: cluster 2024-06-30T13:08:00.360304+0000 mgr.y (mgr.14144) 162 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:01.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:01 smithi144 bash[14758]: audit 2024-06-30T13:08:00.498314+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:01.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:01 smithi144 bash[14758]: audit 2024-06-30T13:08:00.509054+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:01.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:01 smithi144 bash[14758]: audit 2024-06-30T13:08:00.521077+0000 mon.a (mon.0) 402 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:01.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:01 smithi144 bash[19242]: cluster 2024-06-30T13:08:00.360304+0000 mgr.y (mgr.14144) 162 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:01.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:01 smithi144 bash[19242]: audit 2024-06-30T13:08:00.498314+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:01.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:01 smithi144 bash[19242]: audit 2024-06-30T13:08:00.509054+0000 mon.a (mon.0) 401 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:01.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:01 smithi144 bash[19242]: audit 2024-06-30T13:08:00.521077+0000 mon.a (mon.0) 402 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:02.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:02 smithi144 bash[14758]: cluster 2024-06-30T13:08:02.360906+0000 mgr.y (mgr.14144) 163 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:02.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:02 smithi144 bash[19242]: cluster 2024-06-30T13:08:02.360906+0000 mgr.y (mgr.14144) 163 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:02.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:02 smithi167 bash[14619]: cluster 2024-06-30T13:08:02.360906+0000 mgr.y (mgr.14144) 163 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:02.950 INFO:journalctl@ceph.osd.6.smithi167.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:08:03.637 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:03 smithi167 bash[23008]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-06-30T13:08:03.637 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:03 smithi167 bash[23008]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_2 --path /var/lib/ceph/osd/ceph-6 --no-mon-config 2024-06-30T13:08:03.637 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:03 smithi167 bash[23008]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6/block 2024-06-30T13:08:03.637 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:03 smithi167 bash[23008]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-6/block 2024-06-30T13:08:03.637 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:03 smithi167 bash[23008]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-06-30T13:08:03.637 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:03 smithi167 bash[23008]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-06-30T13:08:03.637 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:03 smithi167 bash[23008]: --> ceph-volume lvm activate successful for osd ID: 6 2024-06-30T13:08:04.991 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:04 smithi167 bash[14619]: audit 2024-06-30T13:08:03.685472+0000 mon.a (mon.0) 403 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:04.991 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:04 smithi167 bash[14619]: cluster 2024-06-30T13:08:04.361490+0000 mgr.y (mgr.14144) 164 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:05.008 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:04 smithi144 bash[14758]: audit 2024-06-30T13:08:03.685472+0000 mon.a (mon.0) 403 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:05.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:04 smithi144 bash[14758]: cluster 2024-06-30T13:08:04.361490+0000 mgr.y (mgr.14144) 164 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:05.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:04 smithi144 bash[19242]: audit 2024-06-30T13:08:03.685472+0000 mon.a (mon.0) 403 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:05.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:04 smithi144 bash[19242]: cluster 2024-06-30T13:08:04.361490+0000 mgr.y (mgr.14144) 164 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:07.026 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:06 smithi167 bash[23355]: debug 2024-06-30T13:08:06.969+0000 7fed1626d700 1 -- 172.21.15.167:0/1451369139 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5645050f24e0 con 0x56450516b000 2024-06-30T13:08:07.437 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:07 smithi167 bash[14619]: cluster 2024-06-30T13:08:06.362102+0000 mgr.y (mgr.14144) 165 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:07.747 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:07 smithi167 bash[23355]: debug 2024-06-30T13:08:07.525+0000 7fed1dcf8080 -1 Falling back to public interface 2024-06-30T13:08:07.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:07 smithi144 bash[14758]: cluster 2024-06-30T13:08:06.362102+0000 mgr.y (mgr.14144) 165 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:07.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:07 smithi144 bash[19242]: cluster 2024-06-30T13:08:06.362102+0000 mgr.y (mgr.14144) 165 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:08.167 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-30T13:08:08.167 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-30T13:08:08.167 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-06-30T13:08:08.167 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-06-30T13:08:08.167 INFO:teuthology.orchestra.run.smithi167.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0181317 s, 578 MB/s 2024-06-30T13:08:08.167 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-06-30T13:08:09.129 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-06-30T13:08:09.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:09 smithi144 bash[14758]: cluster 2024-06-30T13:08:08.362676+0000 mgr.y (mgr.14144) 166 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:09.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:09 smithi144 bash[19242]: cluster 2024-06-30T13:08:08.362676+0000 mgr.y (mgr.14144) 166 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:09.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:09 smithi167 bash[14619]: cluster 2024-06-30T13:08:08.362676+0000 mgr.y (mgr.14144) 166 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:10.744 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:10 smithi167 bash[23355]: debug 2024-06-30T13:08:10.597+0000 7fed1dcf8080 -1 osd.6 0 log_to_monitors {default=true} 2024-06-30T13:08:11.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:11 smithi167 bash[14619]: cluster 2024-06-30T13:08:10.363271+0000 mgr.y (mgr.14144) 167 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:11.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:11 smithi167 bash[14619]: audit 2024-06-30T13:08:10.606462+0000 mon.a (mon.0) 404 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-30T13:08:11.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:11 smithi167 bash[14619]: audit 2024-06-30T13:08:10.606802+0000 mon.b (mon.1) 17 : audit [INF] from='osd.6 [v2:172.21.15.167:6816/4265917089,v1:172.21.15.167:6817/4265917089]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-30T13:08:11.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:11 smithi144 bash[14758]: cluster 2024-06-30T13:08:10.363271+0000 mgr.y (mgr.14144) 167 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:11.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:11 smithi144 bash[14758]: audit 2024-06-30T13:08:10.606462+0000 mon.a (mon.0) 404 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-30T13:08:11.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:11 smithi144 bash[14758]: audit 2024-06-30T13:08:10.606802+0000 mon.b (mon.1) 17 : audit [INF] from='osd.6 [v2:172.21.15.167:6816/4265917089,v1:172.21.15.167:6817/4265917089]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-30T13:08:11.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:11 smithi144 bash[19242]: cluster 2024-06-30T13:08:10.363271+0000 mgr.y (mgr.14144) 167 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:11.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:11 smithi144 bash[19242]: audit 2024-06-30T13:08:10.606462+0000 mon.a (mon.0) 404 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-30T13:08:11.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:11 smithi144 bash[19242]: audit 2024-06-30T13:08:10.606802+0000 mon.b (mon.1) 17 : audit [INF] from='osd.6 [v2:172.21.15.167:6816/4265917089,v1:172.21.15.167:6817/4265917089]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-30T13:08:12.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[14758]: audit 2024-06-30T13:08:11.432645+0000 mon.a (mon.0) 405 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-30T13:08:12.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[14758]: cluster 2024-06-30T13:08:11.432919+0000 mon.a (mon.0) 406 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-06-30T13:08:12.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[14758]: audit 2024-06-30T13:08:11.433934+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:12.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[14758]: audit 2024-06-30T13:08:11.435208+0000 mon.a (mon.0) 408 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:12.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[14758]: audit 2024-06-30T13:08:11.435363+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 [v2:172.21.15.167:6816/4265917089,v1:172.21.15.167:6817/4265917089]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:12.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[14758]: audit 2024-06-30T13:08:12.008391+0000 mon.a (mon.0) 409 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:12.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[14758]: audit 2024-06-30T13:08:12.016564+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[14758]: audit 2024-06-30T13:08:12.020536+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[14758]: audit 2024-06-30T13:08:12.028232+0000 mon.a (mon.0) 412 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[19242]: audit 2024-06-30T13:08:11.432645+0000 mon.a (mon.0) 405 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[19242]: cluster 2024-06-30T13:08:11.432919+0000 mon.a (mon.0) 406 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[19242]: audit 2024-06-30T13:08:11.433934+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[19242]: audit 2024-06-30T13:08:11.435208+0000 mon.a (mon.0) 408 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[19242]: audit 2024-06-30T13:08:11.435363+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 [v2:172.21.15.167:6816/4265917089,v1:172.21.15.167:6817/4265917089]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[19242]: audit 2024-06-30T13:08:12.008391+0000 mon.a (mon.0) 409 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[19242]: audit 2024-06-30T13:08:12.016564+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[19242]: audit 2024-06-30T13:08:12.020536+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:12.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:12 smithi144 bash[19242]: audit 2024-06-30T13:08:12.028232+0000 mon.a (mon.0) 412 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:12.776 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[23355]: debug 2024-06-30T13:08:12.441+0000 7fed14269700 -1 osd.6 0 waiting for initial osdmap 2024-06-30T13:08:12.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[14619]: audit 2024-06-30T13:08:11.432645+0000 mon.a (mon.0) 405 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-30T13:08:12.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[14619]: cluster 2024-06-30T13:08:11.432919+0000 mon.a (mon.0) 406 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-06-30T13:08:12.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[14619]: audit 2024-06-30T13:08:11.433934+0000 mon.a (mon.0) 407 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:12.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[14619]: audit 2024-06-30T13:08:11.435208+0000 mon.a (mon.0) 408 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:12.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[14619]: audit 2024-06-30T13:08:11.435363+0000 mon.b (mon.1) 18 : audit [INF] from='osd.6 [v2:172.21.15.167:6816/4265917089,v1:172.21.15.167:6817/4265917089]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:12.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[14619]: audit 2024-06-30T13:08:12.008391+0000 mon.a (mon.0) 409 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:12.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[14619]: audit 2024-06-30T13:08:12.016564+0000 mon.a (mon.0) 410 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:12.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[14619]: audit 2024-06-30T13:08:12.020536+0000 mon.a (mon.0) 411 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:12.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:12 smithi167 bash[14619]: audit 2024-06-30T13:08:12.028232+0000 mon.a (mon.0) 412 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:13.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: cephadm 2024-06-30T13:08:11.998483+0000 mgr.y (mgr.14144) 168 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:08:13.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: cluster 2024-06-30T13:08:12.363928+0000 mgr.y (mgr.14144) 169 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:13.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: audit 2024-06-30T13:08:12.436496+0000 mon.a (mon.0) 413 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:08:13.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: cluster 2024-06-30T13:08:12.436741+0000 mon.a (mon.0) 414 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-06-30T13:08:13.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: audit 2024-06-30T13:08:12.437091+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:13.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: audit 2024-06-30T13:08:12.446727+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: audit 2024-06-30T13:08:13.030576+0000 mgr.y (mgr.14144) 170 : audit [DBG] from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: audit 2024-06-30T13:08:13.032623+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: cephadm 2024-06-30T13:08:13.035747+0000 mgr.y (mgr.14144) 171 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: audit 2024-06-30T13:08:13.036322+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[14758]: audit 2024-06-30T13:08:13.037978+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: cephadm 2024-06-30T13:08:11.998483+0000 mgr.y (mgr.14144) 168 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: cluster 2024-06-30T13:08:12.363928+0000 mgr.y (mgr.14144) 169 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: audit 2024-06-30T13:08:12.436496+0000 mon.a (mon.0) 413 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: cluster 2024-06-30T13:08:12.436741+0000 mon.a (mon.0) 414 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: audit 2024-06-30T13:08:12.437091+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: audit 2024-06-30T13:08:12.446727+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:13.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: audit 2024-06-30T13:08:13.030576+0000 mgr.y (mgr.14144) 170 : audit [DBG] from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:08:13.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: audit 2024-06-30T13:08:13.032623+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:08:13.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: cephadm 2024-06-30T13:08:13.035747+0000 mgr.y (mgr.14144) 171 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:08:13.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: audit 2024-06-30T13:08:13.036322+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:08:13.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:13 smithi144 bash[19242]: audit 2024-06-30T13:08:13.037978+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: cephadm 2024-06-30T13:08:11.998483+0000 mgr.y (mgr.14144) 168 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: cluster 2024-06-30T13:08:12.363928+0000 mgr.y (mgr.14144) 169 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: audit 2024-06-30T13:08:12.436496+0000 mon.a (mon.0) 413 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: cluster 2024-06-30T13:08:12.436741+0000 mon.a (mon.0) 414 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: audit 2024-06-30T13:08:12.437091+0000 mon.a (mon.0) 415 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: audit 2024-06-30T13:08:12.446727+0000 mon.a (mon.0) 416 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: audit 2024-06-30T13:08:13.030576+0000 mgr.y (mgr.14144) 170 : audit [DBG] from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: audit 2024-06-30T13:08:13.032623+0000 mon.a (mon.0) 417 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: cephadm 2024-06-30T13:08:13.035747+0000 mgr.y (mgr.14144) 171 : cephadm [INF] Applying service osd.None on host smithi167... 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: audit 2024-06-30T13:08:13.036322+0000 mon.a (mon.0) 418 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-30T13:08:13.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:13 smithi167 bash[14619]: audit 2024-06-30T13:08:13.037978+0000 mon.a (mon.0) 419 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:08:14.742 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:14 smithi167 bash[14619]: cluster 2024-06-30T13:08:11.589625+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:08:14.743 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:14 smithi167 bash[14619]: cluster 2024-06-30T13:08:11.589794+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:08:14.743 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:14 smithi167 bash[14619]: audit 2024-06-30T13:08:13.443856+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:14.743 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:14 smithi167 bash[14619]: cluster 2024-06-30T13:08:13.452383+0000 mon.a (mon.0) 421 : cluster [INF] osd.6 [v2:172.21.15.167:6816/4265917089,v1:172.21.15.167:6817/4265917089] boot 2024-06-30T13:08:14.743 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:14 smithi167 bash[14619]: cluster 2024-06-30T13:08:13.452496+0000 mon.a (mon.0) 422 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2024-06-30T13:08:14.743 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:14 smithi167 bash[14619]: audit 2024-06-30T13:08:13.453648+0000 mon.a (mon.0) 423 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:14.743 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:14 smithi167 bash[14619]: cluster 2024-06-30T13:08:14.364526+0000 mgr.y (mgr.14144) 172 : cluster [DBG] pgmap v161: 1 pgs: 1 peering; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:14.743 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:14 smithi167 bash[14619]: cluster 2024-06-30T13:08:14.406511+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[14758]: cluster 2024-06-30T13:08:11.589625+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[14758]: cluster 2024-06-30T13:08:11.589794+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[14758]: audit 2024-06-30T13:08:13.443856+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[14758]: cluster 2024-06-30T13:08:13.452383+0000 mon.a (mon.0) 421 : cluster [INF] osd.6 [v2:172.21.15.167:6816/4265917089,v1:172.21.15.167:6817/4265917089] boot 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[14758]: cluster 2024-06-30T13:08:13.452496+0000 mon.a (mon.0) 422 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[14758]: audit 2024-06-30T13:08:13.453648+0000 mon.a (mon.0) 423 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[14758]: cluster 2024-06-30T13:08:14.364526+0000 mgr.y (mgr.14144) 172 : cluster [DBG] pgmap v161: 1 pgs: 1 peering; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[14758]: cluster 2024-06-30T13:08:14.406511+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[19242]: cluster 2024-06-30T13:08:11.589625+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[19242]: cluster 2024-06-30T13:08:11.589794+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:08:14.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[19242]: audit 2024-06-30T13:08:13.443856+0000 mon.a (mon.0) 420 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:14.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[19242]: cluster 2024-06-30T13:08:13.452383+0000 mon.a (mon.0) 421 : cluster [INF] osd.6 [v2:172.21.15.167:6816/4265917089,v1:172.21.15.167:6817/4265917089] boot 2024-06-30T13:08:14.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[19242]: cluster 2024-06-30T13:08:13.452496+0000 mon.a (mon.0) 422 : cluster [DBG] osdmap e42: 7 total, 7 up, 7 in 2024-06-30T13:08:14.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[19242]: audit 2024-06-30T13:08:13.453648+0000 mon.a (mon.0) 423 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-30T13:08:14.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[19242]: cluster 2024-06-30T13:08:14.364526+0000 mgr.y (mgr.14144) 172 : cluster [DBG] pgmap v161: 1 pgs: 1 peering; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2024-06-30T13:08:14.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:14 smithi144 bash[19242]: cluster 2024-06-30T13:08:14.406511+0000 mon.a (mon.0) 424 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-06-30T13:08:16.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:16 smithi144 bash[14758]: cluster 2024-06-30T13:08:15.415986+0000 mon.a (mon.0) 425 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-06-30T13:08:16.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:16 smithi144 bash[19242]: cluster 2024-06-30T13:08:15.415986+0000 mon.a (mon.0) 425 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-06-30T13:08:16.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:16 smithi167 bash[14619]: cluster 2024-06-30T13:08:15.415986+0000 mon.a (mon.0) 425 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-06-30T13:08:17.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[14758]: cluster 2024-06-30T13:08:16.365197+0000 mgr.y (mgr.14144) 173 : cluster [DBG] pgmap v164: 1 pgs: 1 peering; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:17.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[14758]: audit 2024-06-30T13:08:17.194837+0000 mon.a (mon.0) 426 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f1e8060-79b4-45b9-a7b4-8b605eb47afd"}]: dispatch 2024-06-30T13:08:17.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[14758]: audit 2024-06-30T13:08:17.195218+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.167:0/773335107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f1e8060-79b4-45b9-a7b4-8b605eb47afd"}]: dispatch 2024-06-30T13:08:17.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[14758]: audit 2024-06-30T13:08:17.203812+0000 mon.a (mon.0) 427 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f1e8060-79b4-45b9-a7b4-8b605eb47afd"}]': finished 2024-06-30T13:08:17.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[14758]: cluster 2024-06-30T13:08:17.203960+0000 mon.a (mon.0) 428 : cluster [DBG] osdmap e45: 8 total, 7 up, 8 in 2024-06-30T13:08:17.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[14758]: audit 2024-06-30T13:08:17.204292+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:17.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[19242]: cluster 2024-06-30T13:08:16.365197+0000 mgr.y (mgr.14144) 173 : cluster [DBG] pgmap v164: 1 pgs: 1 peering; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:17.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[19242]: audit 2024-06-30T13:08:17.194837+0000 mon.a (mon.0) 426 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f1e8060-79b4-45b9-a7b4-8b605eb47afd"}]: dispatch 2024-06-30T13:08:17.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[19242]: audit 2024-06-30T13:08:17.195218+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.167:0/773335107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f1e8060-79b4-45b9-a7b4-8b605eb47afd"}]: dispatch 2024-06-30T13:08:17.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[19242]: audit 2024-06-30T13:08:17.203812+0000 mon.a (mon.0) 427 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f1e8060-79b4-45b9-a7b4-8b605eb47afd"}]': finished 2024-06-30T13:08:17.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[19242]: cluster 2024-06-30T13:08:17.203960+0000 mon.a (mon.0) 428 : cluster [DBG] osdmap e45: 8 total, 7 up, 8 in 2024-06-30T13:08:17.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:17 smithi144 bash[19242]: audit 2024-06-30T13:08:17.204292+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:17.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:17 smithi167 bash[14619]: cluster 2024-06-30T13:08:16.365197+0000 mgr.y (mgr.14144) 173 : cluster [DBG] pgmap v164: 1 pgs: 1 peering; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:17.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:17 smithi167 bash[14619]: audit 2024-06-30T13:08:17.194837+0000 mon.a (mon.0) 426 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f1e8060-79b4-45b9-a7b4-8b605eb47afd"}]: dispatch 2024-06-30T13:08:17.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:17 smithi167 bash[14619]: audit 2024-06-30T13:08:17.195218+0000 mon.b (mon.1) 19 : audit [INF] from='client.? 172.21.15.167:0/773335107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5f1e8060-79b4-45b9-a7b4-8b605eb47afd"}]: dispatch 2024-06-30T13:08:17.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:17 smithi167 bash[14619]: audit 2024-06-30T13:08:17.203812+0000 mon.a (mon.0) 427 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f1e8060-79b4-45b9-a7b4-8b605eb47afd"}]': finished 2024-06-30T13:08:17.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:17 smithi167 bash[14619]: cluster 2024-06-30T13:08:17.203960+0000 mon.a (mon.0) 428 : cluster [DBG] osdmap e45: 8 total, 7 up, 8 in 2024-06-30T13:08:17.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:17 smithi167 bash[14619]: audit 2024-06-30T13:08:17.204292+0000 mon.a (mon.0) 429 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:18.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:18 smithi144 bash[14758]: audit 2024-06-30T13:08:17.836846+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? 172.21.15.167:0/525510598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:08:18.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:18 smithi144 bash[19242]: audit 2024-06-30T13:08:17.836846+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? 172.21.15.167:0/525510598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:08:18.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:18 smithi167 bash[14619]: audit 2024-06-30T13:08:17.836846+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? 172.21.15.167:0/525510598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-30T13:08:19.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:19 smithi144 bash[14758]: cluster 2024-06-30T13:08:18.365743+0000 mgr.y (mgr.14144) 174 : cluster [DBG] pgmap v166: 1 pgs: 1 peering; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:19.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:19 smithi144 bash[19242]: cluster 2024-06-30T13:08:18.365743+0000 mgr.y (mgr.14144) 174 : cluster [DBG] pgmap v166: 1 pgs: 1 peering; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:19.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:19 smithi167 bash[14619]: cluster 2024-06-30T13:08:18.365743+0000 mgr.y (mgr.14144) 174 : cluster [DBG] pgmap v166: 1 pgs: 1 peering; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:21.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:21 smithi167 bash[14619]: cluster 2024-06-30T13:08:20.366448+0000 mgr.y (mgr.14144) 175 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:21.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:21 smithi144 bash[14758]: cluster 2024-06-30T13:08:20.366448+0000 mgr.y (mgr.14144) 175 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:21.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:21 smithi144 bash[19242]: cluster 2024-06-30T13:08:20.366448+0000 mgr.y (mgr.14144) 175 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:22.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:22 smithi144 bash[14758]: cluster 2024-06-30T13:08:22.367056+0000 mgr.y (mgr.14144) 176 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:22.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:22 smithi144 bash[19242]: cluster 2024-06-30T13:08:22.367056+0000 mgr.y (mgr.14144) 176 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:22.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:22 smithi167 bash[14619]: cluster 2024-06-30T13:08:22.367056+0000 mgr.y (mgr.14144) 176 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:25.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:25 smithi144 bash[14758]: cluster 2024-06-30T13:08:24.367631+0000 mgr.y (mgr.14144) 177 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:25.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:25 smithi144 bash[14758]: audit 2024-06-30T13:08:25.306933+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-30T13:08:25.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:25 smithi144 bash[14758]: audit 2024-06-30T13:08:25.308979+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:08:25.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:25 smithi144 bash[19242]: cluster 2024-06-30T13:08:24.367631+0000 mgr.y (mgr.14144) 177 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:25.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:25 smithi144 bash[19242]: audit 2024-06-30T13:08:25.306933+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-30T13:08:25.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:25 smithi144 bash[19242]: audit 2024-06-30T13:08:25.308979+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:08:25.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:25 smithi167 bash[14619]: cluster 2024-06-30T13:08:24.367631+0000 mgr.y (mgr.14144) 177 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:25.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:25 smithi167 bash[14619]: audit 2024-06-30T13:08:25.306933+0000 mon.a (mon.0) 430 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-30T13:08:25.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:25 smithi167 bash[14619]: audit 2024-06-30T13:08:25.308979+0000 mon.a (mon.0) 431 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:08:26.750 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:26 smithi167 bash[14619]: cephadm 2024-06-30T13:08:25.310601+0000 mgr.y (mgr.14144) 178 : cephadm [INF] Deploying daemon osd.7 on smithi167 2024-06-30T13:08:26.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:26 smithi144 bash[14758]: cephadm 2024-06-30T13:08:25.310601+0000 mgr.y (mgr.14144) 178 : cephadm [INF] Deploying daemon osd.7 on smithi167 2024-06-30T13:08:26.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:26 smithi144 bash[19242]: cephadm 2024-06-30T13:08:25.310601+0000 mgr.y (mgr.14144) 178 : cephadm [INF] Deploying daemon osd.7 on smithi167 2024-06-30T13:08:27.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:27 smithi144 bash[14758]: cluster 2024-06-30T13:08:26.368208+0000 mgr.y (mgr.14144) 179 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:27.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:27 smithi144 bash[19242]: cluster 2024-06-30T13:08:26.368208+0000 mgr.y (mgr.14144) 179 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:27.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:27 smithi167 bash[14619]: cluster 2024-06-30T13:08:26.368208+0000 mgr.y (mgr.14144) 179 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:27.925 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 7 on host 'smithi167' 2024-06-30T13:08:28.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:28 smithi144 bash[14758]: audit 2024-06-30T13:08:27.903951+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:28.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:28 smithi144 bash[14758]: audit 2024-06-30T13:08:27.912591+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:28.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:28 smithi144 bash[14758]: audit 2024-06-30T13:08:27.922680+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:28.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:28 smithi144 bash[19242]: audit 2024-06-30T13:08:27.903951+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:28.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:28 smithi144 bash[19242]: audit 2024-06-30T13:08:27.912591+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:28.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:28 smithi144 bash[19242]: audit 2024-06-30T13:08:27.922680+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:28.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:28 smithi167 bash[14619]: audit 2024-06-30T13:08:27.903951+0000 mon.a (mon.0) 432 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:28.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:28 smithi167 bash[14619]: audit 2024-06-30T13:08:27.912591+0000 mon.a (mon.0) 433 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:28.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:28 smithi167 bash[14619]: audit 2024-06-30T13:08:27.922680+0000 mon.a (mon.0) 434 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:29.262 DEBUG:teuthology.orchestra.run.smithi167:osd.7> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.7.service 2024-06-30T13:08:29.264 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-06-30T13:08:29.264 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd stat -f json 2024-06-30T13:08:29.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:29 smithi167 bash[14619]: cluster 2024-06-30T13:08:28.368794+0000 mgr.y (mgr.14144) 180 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:29.576 INFO:journalctl@ceph.osd.7.smithi167.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:08:29.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:29 smithi144 bash[14758]: cluster 2024-06-30T13:08:28.368794+0000 mgr.y (mgr.14144) 180 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:29.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:29 smithi144 bash[19242]: cluster 2024-06-30T13:08:28.368794+0000 mgr.y (mgr.14144) 180 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:30.277 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:30 smithi167 bash[25722]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-30T13:08:30.277 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:30 smithi167 bash[25722]: Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/vg_nvme/lv_1 --path /var/lib/ceph/osd/ceph-7 --no-mon-config 2024-06-30T13:08:30.277 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:30 smithi167 bash[25722]: Running command: /usr/bin/ln -snf /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-7/block 2024-06-30T13:08:30.277 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:30 smithi167 bash[25722]: Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-7/block 2024-06-30T13:08:30.277 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:30 smithi167 bash[25722]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0 2024-06-30T13:08:30.277 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:30 smithi167 bash[25722]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-06-30T13:08:30.277 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:30 smithi167 bash[25722]: --> ceph-volume lvm activate successful for osd ID: 7 2024-06-30T13:08:31.686 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:31 smithi167 bash[14619]: cluster 2024-06-30T13:08:30.369403+0000 mgr.y (mgr.14144) 181 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:31.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:31 smithi144 bash[14758]: cluster 2024-06-30T13:08:30.369403+0000 mgr.y (mgr.14144) 181 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:31.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:31 smithi144 bash[19242]: cluster 2024-06-30T13:08:30.369403+0000 mgr.y (mgr.14144) 181 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:32.026 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:31 smithi167 bash[25867]: debug 2024-06-30T13:08:31.789+0000 7f56b60e4700 1 -- 172.21.15.167:0/398580170 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c4e670e4e0 con 0x55c4e6787000 2024-06-30T13:08:32.526 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:32 smithi167 bash[25867]: debug 2024-06-30T13:08:32.333+0000 7f56bdb6f080 -1 Falling back to public interface 2024-06-30T13:08:32.783 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:08:33.179 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:32 smithi167 bash[14619]: audit 2024-06-30T13:08:31.867112+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:33.179 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:32 smithi167 bash[14619]: cluster 2024-06-30T13:08:32.369990+0000 mgr.y (mgr.14144) 182 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:33.179 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:32 smithi167 bash[14619]: audit 2024-06-30T13:08:32.781253+0000 mon.a (mon.0) 436 : audit [DBG] from='client.? 172.21.15.144:0/2480989309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-30T13:08:33.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:32 smithi144 bash[19242]: audit 2024-06-30T13:08:31.867112+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:33.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:32 smithi144 bash[19242]: cluster 2024-06-30T13:08:32.369990+0000 mgr.y (mgr.14144) 182 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:33.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:32 smithi144 bash[19242]: audit 2024-06-30T13:08:32.781253+0000 mon.a (mon.0) 436 : audit [DBG] from='client.? 172.21.15.144:0/2480989309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-30T13:08:33.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:32 smithi144 bash[14758]: audit 2024-06-30T13:08:31.867112+0000 mon.a (mon.0) 435 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:33.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:32 smithi144 bash[14758]: cluster 2024-06-30T13:08:32.369990+0000 mgr.y (mgr.14144) 182 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:33.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:32 smithi144 bash[14758]: audit 2024-06-30T13:08:32.781253+0000 mon.a (mon.0) 436 : audit [DBG] from='client.? 172.21.15.144:0/2480989309' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-30T13:08:33.457 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":45,"num_osds":8,"num_up_osds":7,"osd_up_since":1719752893,"num_in_osds":8,"osd_in_since":1719752897,"num_remapped_pgs":0} 2024-06-30T13:08:34.458 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd stat -f json 2024-06-30T13:08:35.705 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:35 smithi144 bash[14758]: cluster 2024-06-30T13:08:34.370552+0000 mgr.y (mgr.14144) 183 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:35.705 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:35 smithi144 bash[14758]: audit 2024-06-30T13:08:35.326117+0000 mon.a (mon.0) 437 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-30T13:08:35.705 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:35 smithi144 bash[14758]: audit 2024-06-30T13:08:35.326637+0000 mon.b (mon.1) 21 : audit [INF] from='osd.7 [v2:172.21.15.167:6824/563049758,v1:172.21.15.167:6825/563049758]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-30T13:08:35.705 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:35 smithi144 bash[19242]: cluster 2024-06-30T13:08:34.370552+0000 mgr.y (mgr.14144) 183 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:35.705 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:35 smithi144 bash[19242]: audit 2024-06-30T13:08:35.326117+0000 mon.a (mon.0) 437 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-30T13:08:35.705 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:35 smithi144 bash[19242]: audit 2024-06-30T13:08:35.326637+0000 mon.b (mon.1) 21 : audit [INF] from='osd.7 [v2:172.21.15.167:6824/563049758,v1:172.21.15.167:6825/563049758]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-30T13:08:35.776 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:35 smithi167 bash[25867]: debug 2024-06-30T13:08:35.317+0000 7f56bdb6f080 -1 osd.7 0 log_to_monitors {default=true} 2024-06-30T13:08:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:35 smithi167 bash[14619]: cluster 2024-06-30T13:08:34.370552+0000 mgr.y (mgr.14144) 183 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:35 smithi167 bash[14619]: audit 2024-06-30T13:08:35.326117+0000 mon.a (mon.0) 437 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-30T13:08:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:35 smithi167 bash[14619]: audit 2024-06-30T13:08:35.326637+0000 mon.b (mon.1) 21 : audit [INF] from='osd.7 [v2:172.21.15.167:6824/563049758,v1:172.21.15.167:6825/563049758]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-30T13:08:36.452 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[14758]: audit 2024-06-30T13:08:35.441730+0000 mon.a (mon.0) 438 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-30T13:08:36.452 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[14758]: cluster 2024-06-30T13:08:35.441846+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-06-30T13:08:36.452 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[14758]: audit 2024-06-30T13:08:35.442006+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:36.452 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[14758]: audit 2024-06-30T13:08:35.443603+0000 mon.a (mon.0) 441 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:36.452 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[14758]: audit 2024-06-30T13:08:35.444188+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 [v2:172.21.15.167:6824/563049758,v1:172.21.15.167:6825/563049758]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:36.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[19242]: audit 2024-06-30T13:08:35.441730+0000 mon.a (mon.0) 438 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-30T13:08:36.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[19242]: cluster 2024-06-30T13:08:35.441846+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-06-30T13:08:36.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[19242]: audit 2024-06-30T13:08:35.442006+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:36.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[19242]: audit 2024-06-30T13:08:35.443603+0000 mon.a (mon.0) 441 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:36.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:36 smithi144 bash[19242]: audit 2024-06-30T13:08:35.444188+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 [v2:172.21.15.167:6824/563049758,v1:172.21.15.167:6825/563049758]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:36.776 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:08:36 smithi167 bash[25867]: debug 2024-06-30T13:08:36.453+0000 7f56b58e3700 -1 osd.7 0 waiting for initial osdmap 2024-06-30T13:08:36.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:36 smithi167 bash[14619]: audit 2024-06-30T13:08:35.441730+0000 mon.a (mon.0) 438 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-30T13:08:36.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:36 smithi167 bash[14619]: cluster 2024-06-30T13:08:35.441846+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-06-30T13:08:36.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:36 smithi167 bash[14619]: audit 2024-06-30T13:08:35.442006+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:36.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:36 smithi167 bash[14619]: audit 2024-06-30T13:08:35.443603+0000 mon.a (mon.0) 441 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:36.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:36 smithi167 bash[14619]: audit 2024-06-30T13:08:35.444188+0000 mon.b (mon.1) 22 : audit [INF] from='osd.7 [v2:172.21.15.167:6824/563049758,v1:172.21.15.167:6825/563049758]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-06-30T13:08:37.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: cluster 2024-06-30T13:08:36.371156+0000 mgr.y (mgr.14144) 184 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:37.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: audit 2024-06-30T13:08:36.446509+0000 mon.a (mon.0) 442 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:08:37.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: cluster 2024-06-30T13:08:36.446845+0000 mon.a (mon.0) 443 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-06-30T13:08:37.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: audit 2024-06-30T13:08:36.447671+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:37.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: audit 2024-06-30T13:08:36.453991+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:37.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: cephadm 2024-06-30T13:08:37.066780+0000 mgr.y (mgr.14144) 185 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: audit 2024-06-30T13:08:37.076428+0000 mon.a (mon.0) 446 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: audit 2024-06-30T13:08:37.084838+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: audit 2024-06-30T13:08:37.089060+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[14758]: audit 2024-06-30T13:08:37.100109+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: cluster 2024-06-30T13:08:36.371156+0000 mgr.y (mgr.14144) 184 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: audit 2024-06-30T13:08:36.446509+0000 mon.a (mon.0) 442 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: cluster 2024-06-30T13:08:36.446845+0000 mon.a (mon.0) 443 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: audit 2024-06-30T13:08:36.447671+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: audit 2024-06-30T13:08:36.453991+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: cephadm 2024-06-30T13:08:37.066780+0000 mgr.y (mgr.14144) 185 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: audit 2024-06-30T13:08:37.076428+0000 mon.a (mon.0) 446 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:37.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: audit 2024-06-30T13:08:37.084838+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:37.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: audit 2024-06-30T13:08:37.089060+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:37.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:37 smithi144 bash[19242]: audit 2024-06-30T13:08:37.100109+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: cluster 2024-06-30T13:08:36.371156+0000 mgr.y (mgr.14144) 184 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: audit 2024-06-30T13:08:36.446509+0000 mon.a (mon.0) 442 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: cluster 2024-06-30T13:08:36.446845+0000 mon.a (mon.0) 443 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: audit 2024-06-30T13:08:36.447671+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: audit 2024-06-30T13:08:36.453991+0000 mon.a (mon.0) 445 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: cephadm 2024-06-30T13:08:37.066780+0000 mgr.y (mgr.14144) 185 : cephadm [INF] Detected new or changed devices on smithi167 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: audit 2024-06-30T13:08:37.076428+0000 mon.a (mon.0) 446 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: audit 2024-06-30T13:08:37.084838+0000 mon.a (mon.0) 447 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: audit 2024-06-30T13:08:37.089060+0000 mon.a (mon.0) 448 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:37 smithi167 bash[14619]: audit 2024-06-30T13:08:37.100109+0000 mon.a (mon.0) 449 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:38.086 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:08:38.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[14758]: cluster 2024-06-30T13:08:36.333593+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:08:38.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[14758]: cluster 2024-06-30T13:08:36.333737+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:08:38.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[14758]: cluster 2024-06-30T13:08:37.463434+0000 mon.a (mon.0) 450 : cluster [INF] osd.7 [v2:172.21.15.167:6824/563049758,v1:172.21.15.167:6825/563049758] boot 2024-06-30T13:08:38.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[14758]: cluster 2024-06-30T13:08:37.463609+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2024-06-30T13:08:38.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[14758]: audit 2024-06-30T13:08:37.464807+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:38.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[14758]: audit 2024-06-30T13:08:38.083926+0000 mon.a (mon.0) 453 : audit [DBG] from='client.? 172.21.15.144:0/2967455446' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-30T13:08:38.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[14758]: cluster 2024-06-30T13:08:38.371954+0000 mgr.y (mgr.14144) 186 : cluster [DBG] pgmap v179: 1 pgs: 1 remapped+peering; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:38.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[19242]: cluster 2024-06-30T13:08:36.333593+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:08:38.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[19242]: cluster 2024-06-30T13:08:36.333737+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:08:38.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[19242]: cluster 2024-06-30T13:08:37.463434+0000 mon.a (mon.0) 450 : cluster [INF] osd.7 [v2:172.21.15.167:6824/563049758,v1:172.21.15.167:6825/563049758] boot 2024-06-30T13:08:38.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[19242]: cluster 2024-06-30T13:08:37.463609+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2024-06-30T13:08:38.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[19242]: audit 2024-06-30T13:08:37.464807+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:38.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[19242]: audit 2024-06-30T13:08:38.083926+0000 mon.a (mon.0) 453 : audit [DBG] from='client.? 172.21.15.144:0/2967455446' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-30T13:08:38.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:38 smithi144 bash[19242]: cluster 2024-06-30T13:08:38.371954+0000 mgr.y (mgr.14144) 186 : cluster [DBG] pgmap v179: 1 pgs: 1 remapped+peering; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:38.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:38 smithi167 bash[14619]: cluster 2024-06-30T13:08:36.333593+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-06-30T13:08:38.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:38 smithi167 bash[14619]: cluster 2024-06-30T13:08:36.333737+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-06-30T13:08:38.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:38 smithi167 bash[14619]: cluster 2024-06-30T13:08:37.463434+0000 mon.a (mon.0) 450 : cluster [INF] osd.7 [v2:172.21.15.167:6824/563049758,v1:172.21.15.167:6825/563049758] boot 2024-06-30T13:08:38.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:38 smithi167 bash[14619]: cluster 2024-06-30T13:08:37.463609+0000 mon.a (mon.0) 451 : cluster [DBG] osdmap e48: 8 total, 8 up, 8 in 2024-06-30T13:08:38.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:38 smithi167 bash[14619]: audit 2024-06-30T13:08:37.464807+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-30T13:08:38.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:38 smithi167 bash[14619]: audit 2024-06-30T13:08:38.083926+0000 mon.a (mon.0) 453 : audit [DBG] from='client.? 172.21.15.144:0/2967455446' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-30T13:08:38.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:38 smithi167 bash[14619]: cluster 2024-06-30T13:08:38.371954+0000 mgr.y (mgr.14144) 186 : cluster [DBG] pgmap v179: 1 pgs: 1 remapped+peering; 0 B data, 36 MiB used, 626 GiB / 626 GiB avail 2024-06-30T13:08:38.791 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1719752917,"num_in_osds":8,"osd_in_since":1719752897,"num_remapped_pgs":1} 2024-06-30T13:08:38.791 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd dump --format=json 2024-06-30T13:08:39.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:39 smithi144 bash[14758]: cluster 2024-06-30T13:08:38.466456+0000 mon.a (mon.0) 454 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-06-30T13:08:39.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:39 smithi144 bash[14758]: cluster 2024-06-30T13:08:38.481526+0000 mon.a (mon.0) 455 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-06-30T13:08:39.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:39 smithi144 bash[19242]: cluster 2024-06-30T13:08:38.466456+0000 mon.a (mon.0) 454 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-06-30T13:08:39.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:39 smithi144 bash[19242]: cluster 2024-06-30T13:08:38.481526+0000 mon.a (mon.0) 455 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-06-30T13:08:39.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:39 smithi167 bash[14619]: cluster 2024-06-30T13:08:38.466456+0000 mon.a (mon.0) 454 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-06-30T13:08:39.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:39 smithi167 bash[14619]: cluster 2024-06-30T13:08:38.481526+0000 mon.a (mon.0) 455 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-06-30T13:08:40.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:40 smithi144 bash[14758]: cluster 2024-06-30T13:08:39.483339+0000 mon.a (mon.0) 456 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-06-30T13:08:40.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:40 smithi144 bash[14758]: cluster 2024-06-30T13:08:40.372742+0000 mgr.y (mgr.14144) 187 : cluster [DBG] pgmap v182: 1 pgs: 1 remapped+peering; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:40.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:40 smithi144 bash[19242]: cluster 2024-06-30T13:08:39.483339+0000 mon.a (mon.0) 456 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-06-30T13:08:40.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:40 smithi144 bash[19242]: cluster 2024-06-30T13:08:40.372742+0000 mgr.y (mgr.14144) 187 : cluster [DBG] pgmap v182: 1 pgs: 1 remapped+peering; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:40.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:40 smithi167 bash[14619]: cluster 2024-06-30T13:08:39.483339+0000 mon.a (mon.0) 456 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-06-30T13:08:40.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:40 smithi167 bash[14619]: cluster 2024-06-30T13:08:40.372742+0000 mgr.y (mgr.14144) 187 : cluster [DBG] pgmap v182: 1 pgs: 1 remapped+peering; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:41.503 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:08:43.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:43 smithi144 bash[14758]: cluster 2024-06-30T13:08:42.373514+0000 mgr.y (mgr.14144) 188 : cluster [DBG] pgmap v183: 1 pgs: 1 remapped+peering; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:43.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:43 smithi144 bash[19242]: cluster 2024-06-30T13:08:42.373514+0000 mgr.y (mgr.14144) 188 : cluster [DBG] pgmap v183: 1 pgs: 1 remapped+peering; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:43.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:43 smithi167 bash[14619]: cluster 2024-06-30T13:08:42.373514+0000 mgr.y (mgr.14144) 188 : cluster [DBG] pgmap v183: 1 pgs: 1 remapped+peering; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:44.818 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:08:44.818 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":50,"fsid":"f6ab1df6-36e0-11ef-bcaa-c7b262605968","created":"2024-06-30T13:02:57.626587+0000","modified":"2024-06-30T13:08:39.475955+0000","last_up_change":"2024-06-30T13:08:37.449310+0000","last_in_change":"2024-06-30T13:08:17.196072+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-06-30T13:06:28.902705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"86cabe81-91cc-44b6-a500-526db10d84ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6803","nonce":2331490331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6805","nonce":2331490331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6809","nonce":2331490331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6807","nonce":2331490331}]},"public_addr":"172.21.15.144:6803/2331490331","cluster_addr":"172.21.15.144:6805/2331490331","heartbeat_back_addr":"172.21.15.144:6809/2331490331","heartbeat_front_addr":"172.21.15.144:6807/2331490331","state":["exists","up"]},{"osd":1,"uuid":"e13d329a-1644-44d9-9f7c-0bbe011e620c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6811","nonce":2344135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6813","nonce":2344135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6817","nonce":2344135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6815","nonce":2344135033}]},"public_addr":"172.21.15.144:6811/2344135033","cluster_addr":"172.21.15.144:6813/2344135033","heartbeat_back_addr":"172.21.15.144:6817/2344135033","heartbeat_front_addr":"172.21.15.144:6815/2344135033","state":["exists","up"]},{"osd":2,"uuid":"ed9aac72-52c3-4c6a-92ae-cd0ad5033be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6819","nonce":12822724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6821","nonce":12822724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6825","nonce":12822724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6823","nonce":12822724}]},"public_addr":"172.21.15.144:6819/12822724","cluster_addr":"172.21.15.144:6821/12822724","heartbeat_back_addr":"172.21.15.144:6825/12822724","heartbeat_front_addr":"172.21.15.144:6823/12822724","state":["exists","up"]},{"osd":3,"uuid":"f6529f96-89af-4874-92c0-6bf4cf3ee6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6827","nonce":22475720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6829","nonce":22475720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6833","nonce":22475720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6831","nonce":22475720}]},"public_addr":"172.21.15.144:6827/22475720","cluster_addr":"172.21.15.144:6829/22475720","heartbeat_back_addr":"172.21.15.144:6833/22475720","heartbeat_front_addr":"172.21.15.144:6831/22475720","state":["exists","up"]},{"osd":4,"uuid":"0c3e70d8-39f6-4dd5-96e5-840e20a18e04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6801","nonce":828413832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6803","nonce":828413832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6807","nonce":828413832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6805","nonce":828413832}]},"public_addr":"172.21.15.167:6801/828413832","cluster_addr":"172.21.15.167:6803/828413832","heartbeat_back_addr":"172.21.15.167:6807/828413832","heartbeat_front_addr":"172.21.15.167:6805/828413832","state":["exists","up"]},{"osd":5,"uuid":"107635de-2830-4c26-915a-04791174406d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6809","nonce":3971163726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6811","nonce":3971163726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6815","nonce":3971163726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6813","nonce":3971163726}]},"public_addr":"172.21.15.167:6809/3971163726","cluster_addr":"172.21.15.167:6811/3971163726","heartbeat_back_addr":"172.21.15.167:6815/3971163726","heartbeat_front_addr":"172.21.15.167:6813/3971163726","state":["exists","up"]},{"osd":6,"uuid":"b80a36b1-805a-4794-8b5b-53ffd9bd873c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6817","nonce":4265917089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6819","nonce":4265917089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6823","nonce":4265917089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6821","nonce":4265917089}]},"public_addr":"172.21.15.167:6817/4265917089","cluster_addr":"172.21.15.167:6819/4265917089","heartbeat_back_addr":"172.21.15.167:6823/4265917089","heartbeat_front_addr":"172.21.15.167:6821/4265917089","state":["exists","up"]},{"osd":7,"uuid":"5f1e8060-79b4-45b9-a7b4-8b605eb47afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6825","nonce":563049758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6827","nonce":563049758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6831","nonce":563049758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6829","nonce":563049758}]},"public_addr":"172.21.15.167:6825/563049758","cluster_addr":"172.21.15.167:6827/563049758","heartbeat_back_addr":"172.21.15.167:6831/563049758","heartbeat_front_addr":"172.21.15.167:6829/563049758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:05:29.924220+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:05:57.631610+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:06:26.729126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:06:54.143798+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:07:18.366453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:07:43.979575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:08:11.589799+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:08:36.333741+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:0/1176498939":"2024-07-01T13:03:56.235040+0000","172.21.15.144:6801/425147376":"2024-07-01T13:03:56.235040+0000","172.21.15.144:6800/425147376":"2024-07-01T13:03:56.235040+0000","172.21.15.144:0/3212281885":"2024-07-01T13:03:56.235040+0000","172.21.15.144:6800/3582734959":"2024-07-01T13:03:26.191023+0000","172.21.15.144:0/4131766777":"2024-07-01T13:03:26.191023+0000","172.21.15.144:6801/3582734959":"2024-07-01T13:03:26.191023+0000","172.21.15.144:0/425274503":"2024-07-01T13:03:26.191023+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-30T13:08:45.485 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-06-30T13:06:28.902705+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}] 2024-06-30T13:08:45.486 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd pool get device_health_metrics pg_num 2024-06-30T13:08:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:45 smithi144 bash[14758]: cluster 2024-06-30T13:08:44.374308+0000 mgr.y (mgr.14144) 189 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:45 smithi144 bash[14758]: cluster 2024-06-30T13:08:44.429342+0000 mon.a (mon.0) 457 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-06-30T13:08:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:45 smithi144 bash[14758]: cluster 2024-06-30T13:08:44.429425+0000 mon.a (mon.0) 458 : cluster [INF] Cluster is now healthy 2024-06-30T13:08:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:45 smithi144 bash[14758]: audit 2024-06-30T13:08:44.814940+0000 mon.a (mon.0) 459 : audit [DBG] from='client.? 172.21.15.144:0/1883580573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-30T13:08:45.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:45 smithi144 bash[19242]: cluster 2024-06-30T13:08:44.374308+0000 mgr.y (mgr.14144) 189 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:45.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:45 smithi144 bash[19242]: cluster 2024-06-30T13:08:44.429342+0000 mon.a (mon.0) 457 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-06-30T13:08:45.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:45 smithi144 bash[19242]: cluster 2024-06-30T13:08:44.429425+0000 mon.a (mon.0) 458 : cluster [INF] Cluster is now healthy 2024-06-30T13:08:45.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:45 smithi144 bash[19242]: audit 2024-06-30T13:08:44.814940+0000 mon.a (mon.0) 459 : audit [DBG] from='client.? 172.21.15.144:0/1883580573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-30T13:08:45.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:45 smithi167 bash[14619]: cluster 2024-06-30T13:08:44.374308+0000 mgr.y (mgr.14144) 189 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:45.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:45 smithi167 bash[14619]: cluster 2024-06-30T13:08:44.429342+0000 mon.a (mon.0) 457 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-06-30T13:08:45.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:45 smithi167 bash[14619]: cluster 2024-06-30T13:08:44.429425+0000 mon.a (mon.0) 458 : cluster [INF] Cluster is now healthy 2024-06-30T13:08:45.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:45 smithi167 bash[14619]: audit 2024-06-30T13:08:44.814940+0000 mon.a (mon.0) 459 : audit [DBG] from='client.? 172.21.15.144:0/1883580573' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-30T13:08:47.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:47 smithi144 bash[14758]: cluster 2024-06-30T13:08:46.375011+0000 mgr.y (mgr.14144) 190 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:47 smithi144 bash[19242]: cluster 2024-06-30T13:08:46.375011+0000 mgr.y (mgr.14144) 190 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:47.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:47 smithi167 bash[14619]: cluster 2024-06-30T13:08:46.375011+0000 mgr.y (mgr.14144) 190 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:48.200 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:08:49.656 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:49 smithi144 bash[14758]: cluster 2024-06-30T13:08:48.375747+0000 mgr.y (mgr.14144) 191 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:49.656 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:49 smithi144 bash[19242]: cluster 2024-06-30T13:08:48.375747+0000 mgr.y (mgr.14144) 191 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:49.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:49 smithi167 bash[14619]: cluster 2024-06-30T13:08:48.375747+0000 mgr.y (mgr.14144) 191 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:51.534 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2024-06-30T13:08:51.730 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:51 smithi144 bash[14758]: cluster 2024-06-30T13:08:50.376442+0000 mgr.y (mgr.14144) 192 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:51.731 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:51 smithi144 bash[19242]: cluster 2024-06-30T13:08:50.376442+0000 mgr.y (mgr.14144) 192 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:51.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:51 smithi167 bash[14619]: cluster 2024-06-30T13:08:50.376442+0000 mgr.y (mgr.14144) 192 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:52.267 INFO:tasks.cephadm:Adding prometheus.a on smithi167 2024-06-30T13:08:52.267 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch apply prometheus '1;smithi167=a' 2024-06-30T13:08:52.538 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:52 smithi167 bash[14619]: audit 2024-06-30T13:08:51.532279+0000 mon.a (mon.0) 460 : audit [DBG] from='client.? 172.21.15.144:0/3604115002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-06-30T13:08:52.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:52 smithi144 bash[14758]: audit 2024-06-30T13:08:51.532279+0000 mon.a (mon.0) 460 : audit [DBG] from='client.? 172.21.15.144:0/3604115002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-06-30T13:08:52.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:52 smithi144 bash[19242]: audit 2024-06-30T13:08:51.532279+0000 mon.a (mon.0) 460 : audit [DBG] from='client.? 172.21.15.144:0/3604115002' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-06-30T13:08:53.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:53 smithi144 bash[14758]: cluster 2024-06-30T13:08:52.377074+0000 mgr.y (mgr.14144) 193 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:53.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:53 smithi144 bash[19242]: cluster 2024-06-30T13:08:52.377074+0000 mgr.y (mgr.14144) 193 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:53.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:53 smithi167 bash[14619]: cluster 2024-06-30T13:08:52.377074+0000 mgr.y (mgr.14144) 193 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:55.526 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:55 smithi167 bash[14619]: cluster 2024-06-30T13:08:54.377652+0000 mgr.y (mgr.14144) 194 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:55.639 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled prometheus update... 2024-06-30T13:08:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:55 smithi144 bash[14758]: cluster 2024-06-30T13:08:54.377652+0000 mgr.y (mgr.14144) 194 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:55.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:55 smithi144 bash[19242]: cluster 2024-06-30T13:08:54.377652+0000 mgr.y (mgr.14144) 194 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:56.380 DEBUG:teuthology.orchestra.run.smithi167:prometheus.a> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@prometheus.a.service 2024-06-30T13:08:56.382 INFO:tasks.cephadm:Adding node-exporter.a on smithi144 2024-06-30T13:08:56.382 INFO:tasks.cephadm:Adding node-exporter.b on smithi167 2024-06-30T13:08:56.382 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch apply node-exporter '2;smithi144=a;smithi167=b' 2024-06-30T13:08:56.585 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:08:57.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[14758]: audit 2024-06-30T13:08:55.627077+0000 mgr.y (mgr.14144) 195 : audit [DBG] from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:08:57.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[14758]: cephadm 2024-06-30T13:08:55.628604+0000 mgr.y (mgr.14144) 196 : cephadm [INF] Saving service prometheus spec with placement smithi167=a;count:1 2024-06-30T13:08:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[14758]: audit 2024-06-30T13:08:55.636067+0000 mon.a (mon.0) 461 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[14758]: audit 2024-06-30T13:08:55.637699+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[14758]: audit 2024-06-30T13:08:55.887919+0000 mon.a (mon.0) 463 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[14758]: cephadm 2024-06-30T13:08:55.906876+0000 mgr.y (mgr.14144) 197 : cephadm [INF] Deploying daemon prometheus.a on smithi167 2024-06-30T13:08:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[14758]: cluster 2024-06-30T13:08:56.378255+0000 mgr.y (mgr.14144) 198 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[14758]: audit 2024-06-30T13:08:56.507493+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:08:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[14758]: audit 2024-06-30T13:08:56.510087+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:08:57.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[19242]: audit 2024-06-30T13:08:55.627077+0000 mgr.y (mgr.14144) 195 : audit [DBG] from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:08:57.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[19242]: cephadm 2024-06-30T13:08:55.628604+0000 mgr.y (mgr.14144) 196 : cephadm [INF] Saving service prometheus spec with placement smithi167=a;count:1 2024-06-30T13:08:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[19242]: audit 2024-06-30T13:08:55.636067+0000 mon.a (mon.0) 461 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[19242]: audit 2024-06-30T13:08:55.637699+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[19242]: audit 2024-06-30T13:08:55.887919+0000 mon.a (mon.0) 463 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[19242]: cephadm 2024-06-30T13:08:55.906876+0000 mgr.y (mgr.14144) 197 : cephadm [INF] Deploying daemon prometheus.a on smithi167 2024-06-30T13:08:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[19242]: cluster 2024-06-30T13:08:56.378255+0000 mgr.y (mgr.14144) 198 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[19242]: audit 2024-06-30T13:08:56.507493+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:08:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:56 smithi144 bash[19242]: audit 2024-06-30T13:08:56.510087+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:08:57.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:56 smithi167 bash[14619]: audit 2024-06-30T13:08:55.627077+0000 mgr.y (mgr.14144) 195 : audit [DBG] from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:08:57.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:56 smithi167 bash[14619]: cephadm 2024-06-30T13:08:55.628604+0000 mgr.y (mgr.14144) 196 : cephadm [INF] Saving service prometheus spec with placement smithi167=a;count:1 2024-06-30T13:08:57.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:56 smithi167 bash[14619]: audit 2024-06-30T13:08:55.636067+0000 mon.a (mon.0) 461 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:57.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:56 smithi167 bash[14619]: audit 2024-06-30T13:08:55.637699+0000 mon.a (mon.0) 462 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:08:57.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:56 smithi167 bash[14619]: audit 2024-06-30T13:08:55.887919+0000 mon.a (mon.0) 463 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:08:57.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:56 smithi167 bash[14619]: cephadm 2024-06-30T13:08:55.906876+0000 mgr.y (mgr.14144) 197 : cephadm [INF] Deploying daemon prometheus.a on smithi167 2024-06-30T13:08:57.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:56 smithi167 bash[14619]: cluster 2024-06-30T13:08:56.378255+0000 mgr.y (mgr.14144) 198 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:57.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:56 smithi167 bash[14619]: audit 2024-06-30T13:08:56.507493+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:08:57.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:56 smithi167 bash[14619]: audit 2024-06-30T13:08:56.510087+0000 mon.a (mon.0) 465 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:08:59.752 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled node-exporter update... 2024-06-30T13:08:59.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:08:59 smithi144 bash[14758]: cluster 2024-06-30T13:08:58.378819+0000 mgr.y (mgr.14144) 199 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:59.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:08:59 smithi144 bash[19242]: cluster 2024-06-30T13:08:58.378819+0000 mgr.y (mgr.14144) 199 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:08:59.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:08:59 smithi167 bash[14619]: cluster 2024-06-30T13:08:58.378819+0000 mgr.y (mgr.14144) 199 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:00.649 DEBUG:teuthology.orchestra.run.smithi144:node-exporter.a> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@node-exporter.a.service 2024-06-30T13:09:00.652 DEBUG:teuthology.orchestra.run.smithi167:node-exporter.b> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@node-exporter.b.service 2024-06-30T13:09:00.654 INFO:tasks.cephadm:Adding alertmanager.a on smithi144 2024-06-30T13:09:00.654 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch apply alertmanager '1;smithi144=a' 2024-06-30T13:09:00.781 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:09:00.851 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:09:00.986 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:00 smithi167 bash[14619]: audit 2024-06-30T13:08:59.741356+0000 mgr.y (mgr.14144) 200 : audit [DBG] from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi144=a;smithi167=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:09:00.986 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:00 smithi167 bash[14619]: cephadm 2024-06-30T13:08:59.742894+0000 mgr.y (mgr.14144) 201 : cephadm [INF] Saving service node-exporter spec with placement smithi144=a;smithi167=b;count:2 2024-06-30T13:09:00.986 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:00 smithi167 bash[14619]: audit 2024-06-30T13:08:59.749998+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:00.986 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:00 smithi167 bash[14619]: cluster 2024-06-30T13:09:00.379408+0000 mgr.y (mgr.14144) 202 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:01.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:00 smithi144 bash[14758]: audit 2024-06-30T13:08:59.741356+0000 mgr.y (mgr.14144) 200 : audit [DBG] from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi144=a;smithi167=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:09:01.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:00 smithi144 bash[14758]: cephadm 2024-06-30T13:08:59.742894+0000 mgr.y (mgr.14144) 201 : cephadm [INF] Saving service node-exporter spec with placement smithi144=a;smithi167=b;count:2 2024-06-30T13:09:01.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:00 smithi144 bash[14758]: audit 2024-06-30T13:08:59.749998+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:01.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:00 smithi144 bash[14758]: cluster 2024-06-30T13:09:00.379408+0000 mgr.y (mgr.14144) 202 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:01.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:00 smithi144 bash[19242]: audit 2024-06-30T13:08:59.741356+0000 mgr.y (mgr.14144) 200 : audit [DBG] from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi144=a;smithi167=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:09:01.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:00 smithi144 bash[19242]: cephadm 2024-06-30T13:08:59.742894+0000 mgr.y (mgr.14144) 201 : cephadm [INF] Saving service node-exporter spec with placement smithi144=a;smithi167=b;count:2 2024-06-30T13:09:01.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:00 smithi144 bash[19242]: audit 2024-06-30T13:08:59.749998+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:01.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:00 smithi144 bash[19242]: cluster 2024-06-30T13:09:00.379408+0000 mgr.y (mgr.14144) 202 : cluster [DBG] pgmap v192: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:03.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:03 smithi144 bash[14758]: cluster 2024-06-30T13:09:02.380072+0000 mgr.y (mgr.14144) 203 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:03.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:03 smithi144 bash[19242]: cluster 2024-06-30T13:09:02.380072+0000 mgr.y (mgr.14144) 203 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:03.776 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:03 smithi167 bash[14619]: cluster 2024-06-30T13:09:02.380072+0000 mgr.y (mgr.14144) 203 : cluster [DBG] pgmap v193: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:05 smithi144 bash[14758]: cluster 2024-06-30T13:09:04.380697+0000 mgr.y (mgr.14144) 204 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:05.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:05 smithi144 bash[19242]: cluster 2024-06-30T13:09:04.380697+0000 mgr.y (mgr.14144) 204 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:05.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:05 smithi167 bash[14619]: cluster 2024-06-30T13:09:04.380697+0000 mgr.y (mgr.14144) 204 : cluster [DBG] pgmap v194: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:06.461 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled alertmanager update... 2024-06-30T13:09:07.515 DEBUG:teuthology.orchestra.run.smithi144:alertmanager.a> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@alertmanager.a.service 2024-06-30T13:09:07.518 INFO:tasks.cephadm:Adding grafana.a on smithi167 2024-06-30T13:09:07.518 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph orch apply grafana '1;smithi167=a' 2024-06-30T13:09:07.648 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:09:07.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:07 smithi144 bash[14758]: cluster 2024-06-30T13:09:06.381335+0000 mgr.y (mgr.14144) 205 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:07.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:07 smithi144 bash[14758]: audit 2024-06-30T13:09:06.442751+0000 mgr.y (mgr.14144) 206 : audit [DBG] from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:09:07.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:07 smithi144 bash[14758]: cephadm 2024-06-30T13:09:06.444393+0000 mgr.y (mgr.14144) 207 : cephadm [INF] Saving service alertmanager spec with placement smithi144=a;count:1 2024-06-30T13:09:07.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:07 smithi144 bash[14758]: audit 2024-06-30T13:09:06.458780+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:07.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:07 smithi144 bash[19242]: cluster 2024-06-30T13:09:06.381335+0000 mgr.y (mgr.14144) 205 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:07.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:07 smithi144 bash[19242]: audit 2024-06-30T13:09:06.442751+0000 mgr.y (mgr.14144) 206 : audit [DBG] from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:09:07.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:07 smithi144 bash[19242]: cephadm 2024-06-30T13:09:06.444393+0000 mgr.y (mgr.14144) 207 : cephadm [INF] Saving service alertmanager spec with placement smithi144=a;count:1 2024-06-30T13:09:07.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:07 smithi144 bash[19242]: audit 2024-06-30T13:09:06.458780+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:07.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:07 smithi167 bash[14619]: cluster 2024-06-30T13:09:06.381335+0000 mgr.y (mgr.14144) 205 : cluster [DBG] pgmap v195: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:07.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:07 smithi167 bash[14619]: audit 2024-06-30T13:09:06.442751+0000 mgr.y (mgr.14144) 206 : audit [DBG] from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi144=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:09:07.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:07 smithi167 bash[14619]: cephadm 2024-06-30T13:09:06.444393+0000 mgr.y (mgr.14144) 207 : cephadm [INF] Saving service alertmanager spec with placement smithi144=a;count:1 2024-06-30T13:09:07.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:07 smithi167 bash[14619]: audit 2024-06-30T13:09:06.458780+0000 mon.a (mon.0) 467 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:08.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:08 smithi144 bash[14758]: cluster 2024-06-30T13:09:08.381922+0000 mgr.y (mgr.14144) 208 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:08.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:08 smithi144 bash[19242]: cluster 2024-06-30T13:09:08.381922+0000 mgr.y (mgr.14144) 208 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:08.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:08 smithi167 bash[14619]: cluster 2024-06-30T13:09:08.381922+0000 mgr.y (mgr.14144) 208 : cluster [DBG] pgmap v196: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:11.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:11 smithi144 bash[14758]: cluster 2024-06-30T13:09:10.382510+0000 mgr.y (mgr.14144) 209 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:11.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:11 smithi144 bash[19242]: cluster 2024-06-30T13:09:10.382510+0000 mgr.y (mgr.14144) 209 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:11.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:11 smithi167 bash[14619]: cluster 2024-06-30T13:09:10.382510+0000 mgr.y (mgr.14144) 209 : cluster [DBG] pgmap v197: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:12.512 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled grafana update... 2024-06-30T13:09:13.630 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:13 smithi167 bash[14619]: cluster 2024-06-30T13:09:12.383101+0000 mgr.y (mgr.14144) 210 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:13.630 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:13 smithi167 bash[14619]: audit 2024-06-30T13:09:12.509600+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:13.631 DEBUG:teuthology.orchestra.run.smithi167:grafana.a> sudo journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@grafana.a.service 2024-06-30T13:09:13.633 INFO:tasks.cephadm:Setting up client nodes... 2024-06-30T13:09:13.633 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-30T13:09:13.699 INFO:journalctl@ceph.grafana.a.smithi167.stdout:-- Logs begin at Sun 2022-08-14 12:20:24 UTC. -- 2024-06-30T13:09:13.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:13 smithi144 bash[14758]: cluster 2024-06-30T13:09:12.383101+0000 mgr.y (mgr.14144) 210 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:13.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:13 smithi144 bash[14758]: audit 2024-06-30T13:09:12.509600+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:13.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:13 smithi144 bash[19242]: cluster 2024-06-30T13:09:12.383101+0000 mgr.y (mgr.14144) 210 : cluster [DBG] pgmap v198: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:13.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:13 smithi144 bash[19242]: audit 2024-06-30T13:09:12.509600+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:14.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:14 smithi167 bash[14619]: audit 2024-06-30T13:09:12.501096+0000 mgr.y (mgr.14144) 211 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:09:14.717 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:14 smithi167 bash[14619]: cephadm 2024-06-30T13:09:12.502581+0000 mgr.y (mgr.14144) 212 : cephadm [INF] Saving service grafana spec with placement smithi167=a;count:1 2024-06-30T13:09:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:14 smithi144 bash[14758]: audit 2024-06-30T13:09:12.501096+0000 mgr.y (mgr.14144) 211 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:09:14.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:14 smithi144 bash[14758]: cephadm 2024-06-30T13:09:12.502581+0000 mgr.y (mgr.14144) 212 : cephadm [INF] Saving service grafana spec with placement smithi167=a;count:1 2024-06-30T13:09:14.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:14 smithi144 bash[19242]: audit 2024-06-30T13:09:12.501096+0000 mgr.y (mgr.14144) 211 : audit [DBG] from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi167=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:09:14.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:14 smithi144 bash[19242]: cephadm 2024-06-30T13:09:12.502581+0000 mgr.y (mgr.14144) 212 : cephadm [INF] Saving service grafana spec with placement smithi167=a;count:1 2024-06-30T13:09:15.442 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:15 smithi167 bash[14619]: cluster 2024-06-30T13:09:14.383702+0000 mgr.y (mgr.14144) 213 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:15.697 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:15 smithi167 systemd[1]: Started Ceph prometheus.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:09:15.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:15 smithi144 bash[14758]: cluster 2024-06-30T13:09:14.383702+0000 mgr.y (mgr.14144) 213 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:15.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:15 smithi144 bash[19242]: cluster 2024-06-30T13:09:14.383702+0000 mgr.y (mgr.14144) 213 : cluster [DBG] pgmap v199: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:17.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:16 smithi144 bash[14758]: audit 2024-06-30T13:09:15.625356+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:17.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:16 smithi144 bash[14758]: audit 2024-06-30T13:09:15.626842+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:09:17.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:16 smithi144 bash[14758]: cluster 2024-06-30T13:09:16.384371+0000 mgr.y (mgr.14144) 214 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:17.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:16 smithi144 bash[19242]: audit 2024-06-30T13:09:15.625356+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:17.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:16 smithi144 bash[19242]: audit 2024-06-30T13:09:15.626842+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:09:17.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:16 smithi144 bash[19242]: cluster 2024-06-30T13:09:16.384371+0000 mgr.y (mgr.14144) 214 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:17.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[14619]: audit 2024-06-30T13:09:15.625356+0000 mon.a (mon.0) 469 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:17.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[14619]: audit 2024-06-30T13:09:15.626842+0000 mon.a (mon.0) 470 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:09:17.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[14619]: cluster 2024-06-30T13:09:16.384371+0000 mgr.y (mgr.14144) 214 : cluster [DBG] pgmap v200: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:17.027 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.859Z caller=main.go:302 msg="No time or size retention was set so using the default time retention" duration=15d 2024-06-30T13:09:17.027 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.859Z caller=main.go:337 msg="Starting Prometheus" version="(version=2.18.1, branch=HEAD, revision=ecee9c8abfd118f139014cb1b174b08db3f342cf)" 2024-06-30T13:09:17.027 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.859Z caller=main.go:338 build_context="(go=go1.14.2, user=root@2117a9e64a7e, date=20200507-16:51:47)" 2024-06-30T13:09:17.027 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.859Z caller=main.go:339 host_details="(Linux 5.4.0-187-generic #207-Ubuntu SMP Mon Jun 10 08:16:10 UTC 2024 x86_64 smithi167 (none))" 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.859Z caller=main.go:340 fd_limits="(soft=1048576, hard=1048576)" 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.859Z caller=main.go:341 vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.860Z caller=web.go:523 component=web msg="Start listening for connections" address=:9095 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.860Z caller=main.go:678 msg="Starting TSDB ..." 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.865Z caller=head.go:575 component=tsdb msg="Replaying WAL, this may take awhile" 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.866Z caller=head.go:624 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.866Z caller=head.go:627 component=tsdb msg="WAL replay completed" duration=400.137µs 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.867Z caller=main.go:694 fs_type=XFS_SUPER_MAGIC 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.867Z caller=main.go:695 msg="TSDB started" 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.867Z caller=main.go:799 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.870Z caller=main.go:827 msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-30T13:09:17.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:09:16 smithi167 bash[27779]: level=info ts=2024-06-30T13:09:16.870Z caller=main.go:646 msg="Server is ready to receive web requests." 2024-06-30T13:09:17.493 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2024-06-30T13:09:17.493 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD9WIFm3jm1HBAABFUg++TpE0gC7mt6tqBfQw== 2024-06-30T13:09:18.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:17 smithi144 bash[14758]: audit 2024-06-30T13:09:17.480583+0000 mon.c (mon.2) 8 : audit [INF] from='client.? 172.21.15.144:0/2252939851' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:18.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:17 smithi144 bash[14758]: audit 2024-06-30T13:09:17.481297+0000 mon.a (mon.0) 471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:18.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:17 smithi144 bash[14758]: audit 2024-06-30T13:09:17.490425+0000 mon.a (mon.0) 472 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-30T13:09:18.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:17 smithi144 bash[19242]: audit 2024-06-30T13:09:17.480583+0000 mon.c (mon.2) 8 : audit [INF] from='client.? 172.21.15.144:0/2252939851' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:18.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:17 smithi144 bash[19242]: audit 2024-06-30T13:09:17.481297+0000 mon.a (mon.0) 471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:18.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:17 smithi144 bash[19242]: audit 2024-06-30T13:09:17.490425+0000 mon.a (mon.0) 472 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-30T13:09:18.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:17 smithi167 bash[14619]: audit 2024-06-30T13:09:17.480583+0000 mon.c (mon.2) 8 : audit [INF] from='client.? 172.21.15.144:0/2252939851' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:18.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:17 smithi167 bash[14619]: audit 2024-06-30T13:09:17.481297+0000 mon.a (mon.0) 471 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:18.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:17 smithi167 bash[14619]: audit 2024-06-30T13:09:17.490425+0000 mon.a (mon.0) 472 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-30T13:09:18.190 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-06-30T13:09:18.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-30T13:09:18.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-30T13:09:18.219 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-30T13:09:18.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:18 smithi167 bash[14619]: cluster 2024-06-30T13:09:18.385024+0000 mgr.y (mgr.14144) 215 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:19.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:18 smithi144 bash[14758]: cluster 2024-06-30T13:09:18.385024+0000 mgr.y (mgr.14144) 215 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:19.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:18 smithi144 bash[19242]: cluster 2024-06-30T13:09:18.385024+0000 mgr.y (mgr.14144) 215 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:20.894 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:20 smithi167 systemd[1]: Started Ceph node-exporter.b for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:09:21.277 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:20 smithi167 bash[28225]: Unable to find image 'prom/node-exporter:v0.18.1' locally 2024-06-30T13:09:21.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:21 smithi167 bash[14619]: audit 2024-06-30T13:09:20.033552+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:21.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:21 smithi167 bash[14619]: audit 2024-06-30T13:09:20.043749+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:21.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:21 smithi167 bash[14619]: cephadm 2024-06-30T13:09:20.046830+0000 mgr.y (mgr.14144) 216 : cephadm [INF] Deploying daemon node-exporter.b on smithi167 2024-06-30T13:09:21.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:21 smithi167 bash[14619]: cluster 2024-06-30T13:09:20.385698+0000 mgr.y (mgr.14144) 217 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:21.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:21 smithi167 bash[14619]: audit 2024-06-30T13:09:20.889444+0000 mon.a (mon.0) 475 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:21.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[19242]: audit 2024-06-30T13:09:20.033552+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:21.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[19242]: audit 2024-06-30T13:09:20.043749+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:21.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[19242]: cephadm 2024-06-30T13:09:20.046830+0000 mgr.y (mgr.14144) 216 : cephadm [INF] Deploying daemon node-exporter.b on smithi167 2024-06-30T13:09:21.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[19242]: cluster 2024-06-30T13:09:20.385698+0000 mgr.y (mgr.14144) 217 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:21.354 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[19242]: audit 2024-06-30T13:09:20.889444+0000 mon.a (mon.0) 475 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:21.354 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[14758]: audit 2024-06-30T13:09:20.033552+0000 mon.a (mon.0) 473 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:21.354 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[14758]: audit 2024-06-30T13:09:20.043749+0000 mon.a (mon.0) 474 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:21.355 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[14758]: cephadm 2024-06-30T13:09:20.046830+0000 mgr.y (mgr.14144) 216 : cephadm [INF] Deploying daemon node-exporter.b on smithi167 2024-06-30T13:09:21.355 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[14758]: cluster 2024-06-30T13:09:20.385698+0000 mgr.y (mgr.14144) 217 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:21.355 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[14758]: audit 2024-06-30T13:09:20.889444+0000 mon.a (mon.0) 475 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:21.608 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:21 smithi144 systemd[1]: Started Ceph node-exporter.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:09:21.999 INFO:teuthology.orchestra.run.smithi167.stdout:[client.1] 2024-06-30T13:09:22.000 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQABWYFmOIHzOhAAP01qLw+D1KGaVXJbLn2Wmg== 2024-06-30T13:09:22.009 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:21 smithi144 bash[31929]: Unable to find image 'prom/node-exporter:v0.18.1' locally 2024-06-30T13:09:22.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[14758]: cephadm 2024-06-30T13:09:20.891071+0000 mgr.y (mgr.14144) 218 : cephadm [INF] Deploying daemon node-exporter.a on smithi144 2024-06-30T13:09:22.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[14758]: audit 2024-06-30T13:09:21.602867+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:22.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[14758]: audit 2024-06-30T13:09:21.988665+0000 mon.a (mon.0) 477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:22.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[14758]: audit 2024-06-30T13:09:21.988975+0000 mon.b (mon.1) 23 : audit [INF] from='client.? 172.21.15.167:0/1315815438' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:22.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[14758]: audit 2024-06-30T13:09:21.997120+0000 mon.a (mon.0) 478 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-30T13:09:22.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[19242]: cephadm 2024-06-30T13:09:20.891071+0000 mgr.y (mgr.14144) 218 : cephadm [INF] Deploying daemon node-exporter.a on smithi144 2024-06-30T13:09:22.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[19242]: audit 2024-06-30T13:09:21.602867+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:22.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[19242]: audit 2024-06-30T13:09:21.988665+0000 mon.a (mon.0) 477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:22.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[19242]: audit 2024-06-30T13:09:21.988975+0000 mon.b (mon.1) 23 : audit [INF] from='client.? 172.21.15.167:0/1315815438' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:22.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:22 smithi144 bash[19242]: audit 2024-06-30T13:09:21.997120+0000 mon.a (mon.0) 478 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-30T13:09:22.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:22 smithi167 bash[14619]: cephadm 2024-06-30T13:09:20.891071+0000 mgr.y (mgr.14144) 218 : cephadm [INF] Deploying daemon node-exporter.a on smithi144 2024-06-30T13:09:22.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:22 smithi167 bash[14619]: audit 2024-06-30T13:09:21.602867+0000 mon.a (mon.0) 476 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:22.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:22 smithi167 bash[14619]: audit 2024-06-30T13:09:21.988665+0000 mon.a (mon.0) 477 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:22.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:22 smithi167 bash[14619]: audit 2024-06-30T13:09:21.988975+0000 mon.b (mon.1) 23 : audit [INF] from='client.? 172.21.15.167:0/1315815438' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-30T13:09:22.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:22 smithi167 bash[14619]: audit 2024-06-30T13:09:21.997120+0000 mon.a (mon.0) 478 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-30T13:09:22.749 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-06-30T13:09:22.749 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-06-30T13:09:22.749 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-06-30T13:09:22.776 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-30T13:09:22.776 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-30T13:09:22.776 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph mgr dump --format=json 2024-06-30T13:09:23.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:23 smithi144 bash[14758]: cephadm 2024-06-30T13:09:21.609839+0000 mgr.y (mgr.14144) 219 : cephadm [INF] Deploying daemon alertmanager.a on smithi144 2024-06-30T13:09:23.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:23 smithi144 bash[14758]: cluster 2024-06-30T13:09:22.386314+0000 mgr.y (mgr.14144) 220 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:23.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:23 smithi144 bash[19242]: cephadm 2024-06-30T13:09:21.609839+0000 mgr.y (mgr.14144) 219 : cephadm [INF] Deploying daemon alertmanager.a on smithi144 2024-06-30T13:09:23.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:23 smithi144 bash[19242]: cluster 2024-06-30T13:09:22.386314+0000 mgr.y (mgr.14144) 220 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:23.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:23 smithi167 bash[14619]: cephadm 2024-06-30T13:09:21.609839+0000 mgr.y (mgr.14144) 219 : cephadm [INF] Deploying daemon alertmanager.a on smithi144 2024-06-30T13:09:23.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:23 smithi167 bash[14619]: cluster 2024-06-30T13:09:22.386314+0000 mgr.y (mgr.14144) 220 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:25.027 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:24 smithi167 bash[28225]: v0.18.1: Pulling from prom/node-exporter 2024-06-30T13:09:25.027 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:24 smithi167 bash[28225]: 49a2d53aa1af: Pulling fs layer 2024-06-30T13:09:25.027 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:24 smithi167 bash[28225]: 3589a6efd9ce: Pulling fs layer 2024-06-30T13:09:25.027 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:24 smithi167 bash[28225]: 190160031744: Pulling fs layer 2024-06-30T13:09:25.446 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:25 smithi167 bash[28225]: 3589a6efd9ce: Verifying Checksum 2024-06-30T13:09:25.446 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:25 smithi167 bash[28225]: 3589a6efd9ce: Download complete 2024-06-30T13:09:25.446 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:25 smithi167 bash[28225]: 49a2d53aa1af: Download complete 2024-06-30T13:09:25.446 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:25 smithi167 bash[28225]: 190160031744: Verifying Checksum 2024-06-30T13:09:25.446 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:25 smithi167 bash[28225]: 190160031744: Download complete 2024-06-30T13:09:25.446 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:25 smithi167 bash[28225]: 49a2d53aa1af: Pull complete 2024-06-30T13:09:25.496 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:25.625 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[14758]: cluster 2024-06-30T13:09:24.387009+0000 mgr.y (mgr.14144) 221 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:25.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[19242]: cluster 2024-06-30T13:09:24.387009+0000 mgr.y (mgr.14144) 221 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:25.625 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[31929]: v0.18.1: Pulling from prom/node-exporter 2024-06-30T13:09:25.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:25 smithi167 bash[14619]: cluster 2024-06-30T13:09:24.387009+0000 mgr.y (mgr.14144) 221 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:25.964 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[31929]: 49a2d53aa1af: Pulling fs layer 2024-06-30T13:09:25.965 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[31929]: 3589a6efd9ce: Pulling fs layer 2024-06-30T13:09:25.965 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[31929]: 190160031744: Pulling fs layer 2024-06-30T13:09:25.965 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[31929]: 190160031744: Waiting 2024-06-30T13:09:25.965 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[31929]: 3589a6efd9ce: Waiting 2024-06-30T13:09:25.965 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[31929]: 49a2d53aa1af: Waiting 2024-06-30T13:09:26.239 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:25 smithi167 bash[28225]: 3589a6efd9ce: Pull complete 2024-06-30T13:09:26.259 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[31929]: 49a2d53aa1af: Verifying Checksum 2024-06-30T13:09:26.259 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:25 smithi144 bash[31929]: 49a2d53aa1af: Download complete 2024-06-30T13:09:26.259 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:26 smithi144 bash[31929]: 3589a6efd9ce: Verifying Checksum 2024-06-30T13:09:26.259 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:26 smithi144 bash[31929]: 3589a6efd9ce: Download complete 2024-06-30T13:09:26.496 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:26 smithi167 bash[28225]: 190160031744: Pull complete 2024-06-30T13:09:26.496 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:26 smithi167 bash[28225]: Digest: sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee 2024-06-30T13:09:26.496 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:26 smithi167 bash[28225]: Status: Downloaded newer image for prom/node-exporter:v0.18.1 2024-06-30T13:09:26.759 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:26 smithi144 bash[31929]: 190160031744: Verifying Checksum 2024-06-30T13:09:26.759 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:26 smithi144 bash[31929]: 190160031744: Download complete 2024-06-30T13:09:27.259 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:26 smithi144 bash[31929]: 49a2d53aa1af: Pull complete 2024-06-30T13:09:27.710 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[14619]: cluster 2024-06-30T13:09:26.387627+0000 mgr.y (mgr.14144) 222 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:27.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:27 smithi144 bash[14758]: cluster 2024-06-30T13:09:26.387627+0000 mgr.y (mgr.14144) 222 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:27.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:27 smithi144 bash[19242]: cluster 2024-06-30T13:09:26.387627+0000 mgr.y (mgr.14144) 222 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:28.027 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg="Starting node_exporter (version=0.18.1, branch=HEAD, revision=3db77732e925c08f675d7404a8c46466b2ece83e)" source="node_exporter.go:156" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg="Build context (go=go1.12.5, user=root@b50852a1acba, date=20190604-16:41:18)" source="node_exporter.go:157" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg="Enabled collectors:" source="node_exporter.go:97" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - arp" source="node_exporter.go:104" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - bcache" source="node_exporter.go:104" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - bonding" source="node_exporter.go:104" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - conntrack" source="node_exporter.go:104" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - cpu" source="node_exporter.go:104" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - cpufreq" source="node_exporter.go:104" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - diskstats" source="node_exporter.go:104" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - edac" source="node_exporter.go:104" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - entropy" source="node_exporter.go:104" 2024-06-30T13:09:28.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - filefd" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - filesystem" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - hwmon" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - infiniband" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - ipvs" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - loadavg" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - mdadm" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - meminfo" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - netclass" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - netdev" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - netstat" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - nfs" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - nfsd" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - pressure" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - sockstat" source="node_exporter.go:104" 2024-06-30T13:09:28.029 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - stat" source="node_exporter.go:104" 2024-06-30T13:09:28.030 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - textfile" source="node_exporter.go:104" 2024-06-30T13:09:28.030 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - time" source="node_exporter.go:104" 2024-06-30T13:09:28.030 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - uname" source="node_exporter.go:104" 2024-06-30T13:09:28.030 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - vmstat" source="node_exporter.go:104" 2024-06-30T13:09:28.030 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - xfs" source="node_exporter.go:104" 2024-06-30T13:09:28.030 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg=" - zfs" source="node_exporter.go:104" 2024-06-30T13:09:28.030 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:09:27 smithi167 bash[28225]: time="2024-06-30T13:09:27Z" level=info msg="Listening on :9100" source="node_exporter.go:170" 2024-06-30T13:09:28.509 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:28 smithi144 bash[31929]: 3589a6efd9ce: Pull complete 2024-06-30T13:09:29.259 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:29 smithi144 bash[31929]: 190160031744: Pull complete 2024-06-30T13:09:29.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:29 smithi144 bash[14758]: cluster 2024-06-30T13:09:28.388275+0000 mgr.y (mgr.14144) 223 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:29.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:29 smithi144 bash[19242]: cluster 2024-06-30T13:09:28.388275+0000 mgr.y (mgr.14144) 223 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:29.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:29 smithi167 bash[14619]: cluster 2024-06-30T13:09:28.388275+0000 mgr.y (mgr.14144) 223 : cluster [DBG] pgmap v206: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:30.702 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:30 smithi144 bash[14758]: cluster 2024-06-30T13:09:30.388867+0000 mgr.y (mgr.14144) 224 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:30.703 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:30 smithi144 bash[19242]: cluster 2024-06-30T13:09:30.388867+0000 mgr.y (mgr.14144) 224 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:30.703 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:30 smithi144 bash[31929]: Digest: sha256:a2f29256e53cc3e0b64d7a472512600b2e9410347d53cdc85b49f659c17e02ee 2024-06-30T13:09:30.703 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:30 smithi144 bash[31929]: Status: Downloaded newer image for prom/node-exporter:v0.18.1 2024-06-30T13:09:30.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:30 smithi167 bash[14619]: cluster 2024-06-30T13:09:30.388867+0000 mgr.y (mgr.14144) 224 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:32.963 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg="Starting node_exporter (version=0.18.1, branch=HEAD, revision=3db77732e925c08f675d7404a8c46466b2ece83e)" source="node_exporter.go:156" 2024-06-30T13:09:32.963 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg="Build context (go=go1.12.5, user=root@b50852a1acba, date=20190604-16:41:18)" source="node_exporter.go:157" 2024-06-30T13:09:32.963 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg="Enabled collectors:" source="node_exporter.go:97" 2024-06-30T13:09:33.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - arp" source="node_exporter.go:104" 2024-06-30T13:09:33.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - bcache" source="node_exporter.go:104" 2024-06-30T13:09:33.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - bonding" source="node_exporter.go:104" 2024-06-30T13:09:33.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - conntrack" source="node_exporter.go:104" 2024-06-30T13:09:33.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - cpu" source="node_exporter.go:104" 2024-06-30T13:09:33.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - cpufreq" source="node_exporter.go:104" 2024-06-30T13:09:33.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - diskstats" source="node_exporter.go:104" 2024-06-30T13:09:33.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - edac" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - entropy" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - filefd" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - filesystem" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - hwmon" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - infiniband" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - ipvs" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - loadavg" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - mdadm" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - meminfo" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - netclass" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - netdev" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - netstat" source="node_exporter.go:104" 2024-06-30T13:09:33.261 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - nfs" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - nfsd" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - pressure" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - sockstat" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - stat" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - textfile" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - time" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - uname" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - vmstat" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - xfs" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg=" - zfs" source="node_exporter.go:104" 2024-06-30T13:09:33.262 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:09:32 smithi144 bash[31929]: time="2024-06-30T13:09:32Z" level=info msg="Listening on :9100" source="node_exporter.go:170" 2024-06-30T13:09:33.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:33 smithi144 bash[14758]: cluster 2024-06-30T13:09:32.389436+0000 mgr.y (mgr.14144) 225 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:33.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:33 smithi144 bash[19242]: cluster 2024-06-30T13:09:32.389436+0000 mgr.y (mgr.14144) 225 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:33.762 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:09:33.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:33 smithi167 bash[14619]: cluster 2024-06-30T13:09:32.389436+0000 mgr.y (mgr.14144) 225 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:34.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:34 smithi144 bash[14758]: audit 2024-06-30T13:09:33.750269+0000 mon.c (mon.2) 9 : audit [DBG] from='client.? 172.21.15.144:0/702135615' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-30T13:09:34.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:34 smithi144 bash[19242]: audit 2024-06-30T13:09:33.750269+0000 mon.c (mon.2) 9 : audit [DBG] from='client.? 172.21.15.144:0/702135615' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-30T13:09:34.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:34 smithi167 bash[14619]: audit 2024-06-30T13:09:33.750269+0000 mon.c (mon.2) 9 : audit [DBG] from='client.? 172.21.15.144:0/702135615' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-30T13:09:34.896 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":14,"active_gid":14144,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2061007355},{"type":"v1","addr":"172.21.15.144:6801","nonce":2061007355}]},"active_addr":"172.21.15.144:6801/2061007355","active_change":"2024-06-30T13:03:56.235591+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://smithi144.front.sepia.ceph.com:8443/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":3,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":62473058}]},{"addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":2363602481}]}]}} 2024-06-30T13:09:34.901 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-30T13:09:34.902 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-30T13:09:34.902 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd dump --format=json 2024-06-30T13:09:35.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:35 smithi144 bash[14758]: cluster 2024-06-30T13:09:34.390137+0000 mgr.y (mgr.14144) 226 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:35.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:35 smithi144 bash[19242]: cluster 2024-06-30T13:09:34.390137+0000 mgr.y (mgr.14144) 226 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:35 smithi167 bash[14619]: cluster 2024-06-30T13:09:34.390137+0000 mgr.y (mgr.14144) 226 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:37.622 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:37.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:37 smithi144 bash[14758]: cluster 2024-06-30T13:09:36.390792+0000 mgr.y (mgr.14144) 227 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:37.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:37 smithi144 bash[19242]: cluster 2024-06-30T13:09:36.390792+0000 mgr.y (mgr.14144) 227 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:37 smithi167 bash[14619]: cluster 2024-06-30T13:09:36.390792+0000 mgr.y (mgr.14144) 227 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:38.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:38 smithi167 bash[14619]: cluster 2024-06-30T13:09:38.391360+0000 mgr.y (mgr.14144) 228 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:39.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:38 smithi144 bash[14758]: cluster 2024-06-30T13:09:38.391360+0000 mgr.y (mgr.14144) 228 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:39.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:38 smithi144 bash[19242]: cluster 2024-06-30T13:09:38.391360+0000 mgr.y (mgr.14144) 228 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:39.010 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:38 smithi144 systemd[1]: Started Ceph alertmanager.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:09:39.826 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[19242]: audit 2024-06-30T13:09:38.712473+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:39.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[19242]: audit 2024-06-30T13:09:38.909938+0000 mon.a (mon.0) 480 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:39.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[19242]: audit 2024-06-30T13:09:38.918659+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:39.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[19242]: audit 2024-06-30T13:09:38.920014+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-30T13:09:39.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[19242]: audit 2024-06-30T13:09:38.920929+0000 mgr.y (mgr.14144) 229 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-30T13:09:39.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[19242]: audit 2024-06-30T13:09:38.927873+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:39.827 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[19242]: cephadm 2024-06-30T13:09:38.933317+0000 mgr.y (mgr.14144) 230 : cephadm [INF] Deploying daemon grafana.a on smithi167 2024-06-30T13:09:39.827 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[14758]: audit 2024-06-30T13:09:38.712473+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:39.827 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[14758]: audit 2024-06-30T13:09:38.909938+0000 mon.a (mon.0) 480 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:39.827 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[14758]: audit 2024-06-30T13:09:38.918659+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:39.828 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[14758]: audit 2024-06-30T13:09:38.920014+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-30T13:09:39.828 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[14758]: audit 2024-06-30T13:09:38.920929+0000 mgr.y (mgr.14144) 229 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-30T13:09:39.828 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[14758]: audit 2024-06-30T13:09:38.927873+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:39.828 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:39 smithi144 bash[14758]: cephadm 2024-06-30T13:09:38.933317+0000 mgr.y (mgr.14144) 230 : cephadm [INF] Deploying daemon grafana.a on smithi167 2024-06-30T13:09:40.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:39 smithi167 bash[14619]: audit 2024-06-30T13:09:38.712473+0000 mon.a (mon.0) 479 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:40.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:39 smithi167 bash[14619]: audit 2024-06-30T13:09:38.909938+0000 mon.a (mon.0) 480 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:40.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:39 smithi167 bash[14619]: audit 2024-06-30T13:09:38.918659+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:40.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:39 smithi167 bash[14619]: audit 2024-06-30T13:09:38.920014+0000 mon.a (mon.0) 482 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-30T13:09:40.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:39 smithi167 bash[14619]: audit 2024-06-30T13:09:38.920929+0000 mgr.y (mgr.14144) 229 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-30T13:09:40.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:39 smithi167 bash[14619]: audit 2024-06-30T13:09:38.927873+0000 mon.a (mon.0) 483 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:09:40.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:39 smithi167 bash[14619]: cephadm 2024-06-30T13:09:38.933317+0000 mgr.y (mgr.14144) 230 : cephadm [INF] Deploying daemon grafana.a on smithi167 2024-06-30T13:09:41.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:40 smithi144 bash[19242]: cluster 2024-06-30T13:09:40.392050+0000 mgr.y (mgr.14144) 231 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:41.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:40 smithi144 bash[14758]: cluster 2024-06-30T13:09:40.392050+0000 mgr.y (mgr.14144) 231 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:41.010 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:40 smithi144 bash[32518]: level=info ts=2024-06-30T13:09:40.907Z caller=main.go:231 msg="Starting Alertmanager" version="(version=0.20.0, branch=HEAD, revision=f74be0400a6243d10bb53812d6fa408ad71ff32d)" 2024-06-30T13:09:41.010 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:40 smithi144 bash[32518]: level=info ts=2024-06-30T13:09:40.907Z caller=main.go:232 build_context="(go=go1.13.5, user=root@00c3106655f8, date=20191211-14:13:14)" 2024-06-30T13:09:41.010 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:40 smithi144 bash[32518]: level=info ts=2024-06-30T13:09:40.910Z caller=cluster.go:161 component=cluster msg="setting advertise address explicitly" addr=172.21.15.144 port=9094 2024-06-30T13:09:41.011 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:40 smithi144 bash[32518]: level=info ts=2024-06-30T13:09:40.912Z caller=cluster.go:623 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-30T13:09:41.011 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:40 smithi144 bash[32518]: level=info ts=2024-06-30T13:09:40.948Z caller=coordinator.go:119 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-30T13:09:41.011 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:40 smithi144 bash[32518]: level=info ts=2024-06-30T13:09:40.948Z caller=coordinator.go:131 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-30T13:09:41.011 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:40 smithi144 bash[32518]: level=info ts=2024-06-30T13:09:40.952Z caller=main.go:497 msg=Listening address=:9093 2024-06-30T13:09:41.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:40 smithi167 bash[14619]: cluster 2024-06-30T13:09:40.392050+0000 mgr.y (mgr.14144) 231 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:42.519 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:09:42.519 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":50,"fsid":"f6ab1df6-36e0-11ef-bcaa-c7b262605968","created":"2024-06-30T13:02:57.626587+0000","modified":"2024-06-30T13:08:39.475955+0000","last_up_change":"2024-06-30T13:08:37.449310+0000","last_in_change":"2024-06-30T13:08:17.196072+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-06-30T13:06:28.902705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"86cabe81-91cc-44b6-a500-526db10d84ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6803","nonce":2331490331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6805","nonce":2331490331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6809","nonce":2331490331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6807","nonce":2331490331}]},"public_addr":"172.21.15.144:6803/2331490331","cluster_addr":"172.21.15.144:6805/2331490331","heartbeat_back_addr":"172.21.15.144:6809/2331490331","heartbeat_front_addr":"172.21.15.144:6807/2331490331","state":["exists","up"]},{"osd":1,"uuid":"e13d329a-1644-44d9-9f7c-0bbe011e620c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6811","nonce":2344135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6813","nonce":2344135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6817","nonce":2344135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6815","nonce":2344135033}]},"public_addr":"172.21.15.144:6811/2344135033","cluster_addr":"172.21.15.144:6813/2344135033","heartbeat_back_addr":"172.21.15.144:6817/2344135033","heartbeat_front_addr":"172.21.15.144:6815/2344135033","state":["exists","up"]},{"osd":2,"uuid":"ed9aac72-52c3-4c6a-92ae-cd0ad5033be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6819","nonce":12822724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6821","nonce":12822724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6825","nonce":12822724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6823","nonce":12822724}]},"public_addr":"172.21.15.144:6819/12822724","cluster_addr":"172.21.15.144:6821/12822724","heartbeat_back_addr":"172.21.15.144:6825/12822724","heartbeat_front_addr":"172.21.15.144:6823/12822724","state":["exists","up"]},{"osd":3,"uuid":"f6529f96-89af-4874-92c0-6bf4cf3ee6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6827","nonce":22475720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6829","nonce":22475720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6833","nonce":22475720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6831","nonce":22475720}]},"public_addr":"172.21.15.144:6827/22475720","cluster_addr":"172.21.15.144:6829/22475720","heartbeat_back_addr":"172.21.15.144:6833/22475720","heartbeat_front_addr":"172.21.15.144:6831/22475720","state":["exists","up"]},{"osd":4,"uuid":"0c3e70d8-39f6-4dd5-96e5-840e20a18e04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6801","nonce":828413832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6803","nonce":828413832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6807","nonce":828413832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6805","nonce":828413832}]},"public_addr":"172.21.15.167:6801/828413832","cluster_addr":"172.21.15.167:6803/828413832","heartbeat_back_addr":"172.21.15.167:6807/828413832","heartbeat_front_addr":"172.21.15.167:6805/828413832","state":["exists","up"]},{"osd":5,"uuid":"107635de-2830-4c26-915a-04791174406d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6809","nonce":3971163726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6811","nonce":3971163726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6815","nonce":3971163726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6813","nonce":3971163726}]},"public_addr":"172.21.15.167:6809/3971163726","cluster_addr":"172.21.15.167:6811/3971163726","heartbeat_back_addr":"172.21.15.167:6815/3971163726","heartbeat_front_addr":"172.21.15.167:6813/3971163726","state":["exists","up"]},{"osd":6,"uuid":"b80a36b1-805a-4794-8b5b-53ffd9bd873c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6817","nonce":4265917089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6819","nonce":4265917089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6823","nonce":4265917089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6821","nonce":4265917089}]},"public_addr":"172.21.15.167:6817/4265917089","cluster_addr":"172.21.15.167:6819/4265917089","heartbeat_back_addr":"172.21.15.167:6823/4265917089","heartbeat_front_addr":"172.21.15.167:6821/4265917089","state":["exists","up"]},{"osd":7,"uuid":"5f1e8060-79b4-45b9-a7b4-8b605eb47afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6825","nonce":563049758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6827","nonce":563049758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6831","nonce":563049758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6829","nonce":563049758}]},"public_addr":"172.21.15.167:6825/563049758","cluster_addr":"172.21.15.167:6827/563049758","heartbeat_back_addr":"172.21.15.167:6831/563049758","heartbeat_front_addr":"172.21.15.167:6829/563049758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:05:29.924220+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:05:57.631610+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:06:26.729126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:06:54.143798+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:07:18.366453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:07:43.979575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:08:11.589799+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:08:36.333741+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:0/1176498939":"2024-07-01T13:03:56.235040+0000","172.21.15.144:6801/425147376":"2024-07-01T13:03:56.235040+0000","172.21.15.144:6800/425147376":"2024-07-01T13:03:56.235040+0000","172.21.15.144:0/3212281885":"2024-07-01T13:03:56.235040+0000","172.21.15.144:6800/3582734959":"2024-07-01T13:03:26.191023+0000","172.21.15.144:0/4131766777":"2024-07-01T13:03:26.191023+0000","172.21.15.144:6801/3582734959":"2024-07-01T13:03:26.191023+0000","172.21.15.144:0/425274503":"2024-07-01T13:03:26.191023+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-30T13:09:43.009 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:42 smithi144 bash[32518]: level=info ts=2024-06-30T13:09:42.912Z caller=cluster.go:648 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000184999s 2024-06-30T13:09:43.190 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-30T13:09:43.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd dump --format=json 2024-06-30T13:09:43.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:43 smithi144 bash[14758]: cluster 2024-06-30T13:09:42.392500+0000 mgr.y (mgr.14144) 232 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:43.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:43 smithi144 bash[14758]: audit 2024-06-30T13:09:42.515446+0000 mon.a (mon.0) 484 : audit [DBG] from='client.? 172.21.15.144:0/3869936740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-30T13:09:43.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:43 smithi144 bash[19242]: cluster 2024-06-30T13:09:42.392500+0000 mgr.y (mgr.14144) 232 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:43.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:43 smithi144 bash[19242]: audit 2024-06-30T13:09:42.515446+0000 mon.a (mon.0) 484 : audit [DBG] from='client.? 172.21.15.144:0/3869936740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-30T13:09:43.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:43 smithi167 bash[14619]: cluster 2024-06-30T13:09:42.392500+0000 mgr.y (mgr.14144) 232 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:43.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:43 smithi167 bash[14619]: audit 2024-06-30T13:09:42.515446+0000 mon.a (mon.0) 484 : audit [DBG] from='client.? 172.21.15.144:0/3869936740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-30T13:09:45.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:45 smithi144 bash[14758]: cluster 2024-06-30T13:09:44.393134+0000 mgr.y (mgr.14144) 233 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:45.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:45 smithi144 bash[19242]: cluster 2024-06-30T13:09:44.393134+0000 mgr.y (mgr.14144) 233 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:45.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:45 smithi167 bash[14619]: cluster 2024-06-30T13:09:44.393134+0000 mgr.y (mgr.14144) 233 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:45.921 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:46 smithi167 bash[14619]: cluster 2024-06-30T13:09:46.393798+0000 mgr.y (mgr.14144) 234 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:47.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:46 smithi144 bash[14758]: cluster 2024-06-30T13:09:46.393798+0000 mgr.y (mgr.14144) 234 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:47.009 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:46 smithi144 bash[19242]: cluster 2024-06-30T13:09:46.393798+0000 mgr.y (mgr.14144) 234 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:49.454 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:49 smithi144 bash[14758]: cluster 2024-06-30T13:09:48.394454+0000 mgr.y (mgr.14144) 235 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:49.510 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:09:49.575 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":50,"fsid":"f6ab1df6-36e0-11ef-bcaa-c7b262605968","created":"2024-06-30T13:02:57.626587+0000","modified":"2024-06-30T13:08:39.475955+0000","last_up_change":"2024-06-30T13:08:37.449310+0000","last_in_change":"2024-06-30T13:08:17.196072+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-06-30T13:06:28.902705+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"86cabe81-91cc-44b6-a500-526db10d84ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6803","nonce":2331490331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6805","nonce":2331490331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6809","nonce":2331490331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2331490331},{"type":"v1","addr":"172.21.15.144:6807","nonce":2331490331}]},"public_addr":"172.21.15.144:6803/2331490331","cluster_addr":"172.21.15.144:6805/2331490331","heartbeat_back_addr":"172.21.15.144:6809/2331490331","heartbeat_front_addr":"172.21.15.144:6807/2331490331","state":["exists","up"]},{"osd":1,"uuid":"e13d329a-1644-44d9-9f7c-0bbe011e620c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6811","nonce":2344135033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6813","nonce":2344135033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6817","nonce":2344135033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":2344135033},{"type":"v1","addr":"172.21.15.144:6815","nonce":2344135033}]},"public_addr":"172.21.15.144:6811/2344135033","cluster_addr":"172.21.15.144:6813/2344135033","heartbeat_back_addr":"172.21.15.144:6817/2344135033","heartbeat_front_addr":"172.21.15.144:6815/2344135033","state":["exists","up"]},{"osd":2,"uuid":"ed9aac72-52c3-4c6a-92ae-cd0ad5033be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6819","nonce":12822724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6821","nonce":12822724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6825","nonce":12822724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":12822724},{"type":"v1","addr":"172.21.15.144:6823","nonce":12822724}]},"public_addr":"172.21.15.144:6819/12822724","cluster_addr":"172.21.15.144:6821/12822724","heartbeat_back_addr":"172.21.15.144:6825/12822724","heartbeat_front_addr":"172.21.15.144:6823/12822724","state":["exists","up"]},{"osd":3,"uuid":"f6529f96-89af-4874-92c0-6bf4cf3ee6b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6827","nonce":22475720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6829","nonce":22475720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6832","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6833","nonce":22475720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6830","nonce":22475720},{"type":"v1","addr":"172.21.15.144:6831","nonce":22475720}]},"public_addr":"172.21.15.144:6827/22475720","cluster_addr":"172.21.15.144:6829/22475720","heartbeat_back_addr":"172.21.15.144:6833/22475720","heartbeat_front_addr":"172.21.15.144:6831/22475720","state":["exists","up"]},{"osd":4,"uuid":"0c3e70d8-39f6-4dd5-96e5-840e20a18e04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6801","nonce":828413832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6803","nonce":828413832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6807","nonce":828413832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":828413832},{"type":"v1","addr":"172.21.15.167:6805","nonce":828413832}]},"public_addr":"172.21.15.167:6801/828413832","cluster_addr":"172.21.15.167:6803/828413832","heartbeat_back_addr":"172.21.15.167:6807/828413832","heartbeat_front_addr":"172.21.15.167:6805/828413832","state":["exists","up"]},{"osd":5,"uuid":"107635de-2830-4c26-915a-04791174406d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6809","nonce":3971163726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6811","nonce":3971163726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6815","nonce":3971163726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":3971163726},{"type":"v1","addr":"172.21.15.167:6813","nonce":3971163726}]},"public_addr":"172.21.15.167:6809/3971163726","cluster_addr":"172.21.15.167:6811/3971163726","heartbeat_back_addr":"172.21.15.167:6815/3971163726","heartbeat_front_addr":"172.21.15.167:6813/3971163726","state":["exists","up"]},{"osd":6,"uuid":"b80a36b1-805a-4794-8b5b-53ffd9bd873c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6817","nonce":4265917089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6819","nonce":4265917089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6823","nonce":4265917089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":4265917089},{"type":"v1","addr":"172.21.15.167:6821","nonce":4265917089}]},"public_addr":"172.21.15.167:6817/4265917089","cluster_addr":"172.21.15.167:6819/4265917089","heartbeat_back_addr":"172.21.15.167:6823/4265917089","heartbeat_front_addr":"172.21.15.167:6821/4265917089","state":["exists","up"]},{"osd":7,"uuid":"5f1e8060-79b4-45b9-a7b4-8b605eb47afd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6825","nonce":563049758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6827","nonce":563049758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6831","nonce":563049758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":563049758},{"type":"v1","addr":"172.21.15.167:6829","nonce":563049758}]},"public_addr":"172.21.15.167:6825/563049758","cluster_addr":"172.21.15.167:6827/563049758","heartbeat_back_addr":"172.21.15.167:6831/563049758","heartbeat_front_addr":"172.21.15.167:6829/563049758","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:05:29.924220+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:05:57.631610+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:06:26.729126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:06:54.143798+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:07:18.366453+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:07:43.979575+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:08:11.589799+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2024-06-30T13:08:36.333741+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:0/1176498939":"2024-07-01T13:03:56.235040+0000","172.21.15.144:6801/425147376":"2024-07-01T13:03:56.235040+0000","172.21.15.144:6800/425147376":"2024-07-01T13:03:56.235040+0000","172.21.15.144:0/3212281885":"2024-07-01T13:03:56.235040+0000","172.21.15.144:6800/3582734959":"2024-07-01T13:03:26.191023+0000","172.21.15.144:0/4131766777":"2024-07-01T13:03:26.191023+0000","172.21.15.144:6801/3582734959":"2024-07-01T13:03:26.191023+0000","172.21.15.144:0/425274503":"2024-07-01T13:03:26.191023+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-30T13:09:49.712 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:49 smithi144 bash[19242]: cluster 2024-06-30T13:09:48.394454+0000 mgr.y (mgr.14144) 235 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:49.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:49 smithi167 bash[14619]: cluster 2024-06-30T13:09:48.394454+0000 mgr.y (mgr.14144) 235 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:50.230 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-30T13:09:50.231 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-30T13:09:50.231 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-30T13:09:50.231 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-30T13:09:50.232 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-30T13:09:50.232 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-30T13:09:50.233 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-30T13:09:50.233 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-30T13:09:50.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:50 smithi144 bash[14758]: audit 2024-06-30T13:09:49.506874+0000 mon.c (mon.2) 10 : audit [DBG] from='client.? 172.21.15.144:0/3877780003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-30T13:09:50.509 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:50 smithi144 bash[19242]: audit 2024-06-30T13:09:49.506874+0000 mon.c (mon.2) 10 : audit [DBG] from='client.? 172.21.15.144:0/3877780003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-30T13:09:50.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:50 smithi167 bash[14619]: audit 2024-06-30T13:09:49.506874+0000 mon.c (mon.2) 10 : audit [DBG] from='client.? 172.21.15.144:0/3877780003' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-30T13:09:51.259 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:09:50 smithi144 bash[32518]: level=info ts=2024-06-30T13:09:50.913Z caller=cluster.go:640 component=cluster msg="gossip settled; proceeding" elapsed=10.00102822s 2024-06-30T13:09:51.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:51 smithi144 bash[14758]: cluster 2024-06-30T13:09:50.394983+0000 mgr.y (mgr.14144) 236 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:51.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:51 smithi144 bash[19242]: cluster 2024-06-30T13:09:50.394983+0000 mgr.y (mgr.14144) 236 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:51.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:51 smithi167 bash[14619]: cluster 2024-06-30T13:09:50.394983+0000 mgr.y (mgr.14144) 236 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:52.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:52 smithi144 bash[14758]: cluster 2024-06-30T13:09:52.395527+0000 mgr.y (mgr.14144) 237 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:52.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:52 smithi144 bash[19242]: cluster 2024-06-30T13:09:52.395527+0000 mgr.y (mgr.14144) 237 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:52.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:52 smithi167 bash[14619]: cluster 2024-06-30T13:09:52.395527+0000 mgr.y (mgr.14144) 237 : cluster [DBG] pgmap v218: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:53.091 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:53.095 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:53.096 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:53.096 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:53.096 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:53.097 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:53.097 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:53.097 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:09:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:55 smithi144 bash[14758]: cluster 2024-06-30T13:09:54.396106+0000 mgr.y (mgr.14144) 238 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:55.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:55 smithi144 bash[19242]: cluster 2024-06-30T13:09:54.396106+0000 mgr.y (mgr.14144) 238 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:55 smithi167 bash[14619]: cluster 2024-06-30T13:09:54.396106+0000 mgr.y (mgr.14144) 238 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:57.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:57 smithi144 bash[14758]: cluster 2024-06-30T13:09:56.396732+0000 mgr.y (mgr.14144) 239 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:57.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:57 smithi144 bash[14758]: audit 2024-06-30T13:09:56.510512+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:09:57.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:57 smithi144 bash[14758]: audit 2024-06-30T13:09:56.512504+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:09:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:57 smithi144 bash[19242]: cluster 2024-06-30T13:09:56.396732+0000 mgr.y (mgr.14144) 239 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:57 smithi144 bash[19242]: audit 2024-06-30T13:09:56.510512+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:09:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:57 smithi144 bash[19242]: audit 2024-06-30T13:09:56.512504+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:09:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:57 smithi167 bash[14619]: cluster 2024-06-30T13:09:56.396732+0000 mgr.y (mgr.14144) 239 : cluster [DBG] pgmap v220: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:57 smithi167 bash[14619]: audit 2024-06-30T13:09:56.510512+0000 mon.a (mon.0) 485 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:09:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:57 smithi167 bash[14619]: audit 2024-06-30T13:09:56.512504+0000 mon.a (mon.0) 486 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:09:59.627 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:09:59 smithi144 bash[14758]: cluster 2024-06-30T13:09:58.397363+0000 mgr.y (mgr.14144) 240 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:59.627 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:09:59 smithi144 bash[19242]: cluster 2024-06-30T13:09:58.397363+0000 mgr.y (mgr.14144) 240 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:09:59.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:09:59 smithi167 bash[14619]: cluster 2024-06-30T13:09:58.397363+0000 mgr.y (mgr.14144) 240 : cluster [DBG] pgmap v221: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:00.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:00 smithi167 bash[14619]: cluster 2024-06-30T13:10:00.000152+0000 mon.a (mon.0) 487 : cluster [INF] overall HEALTH_OK 2024-06-30T13:10:00.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:00 smithi167 bash[14619]: cluster 2024-06-30T13:10:00.397952+0000 mgr.y (mgr.14144) 241 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:00.867 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:00 smithi144 bash[14758]: cluster 2024-06-30T13:10:00.000152+0000 mon.a (mon.0) 487 : cluster [INF] overall HEALTH_OK 2024-06-30T13:10:00.867 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:00 smithi144 bash[14758]: cluster 2024-06-30T13:10:00.397952+0000 mgr.y (mgr.14144) 241 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:00.867 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:00 smithi144 bash[19242]: cluster 2024-06-30T13:10:00.000152+0000 mon.a (mon.0) 487 : cluster [INF] overall HEALTH_OK 2024-06-30T13:10:00.867 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:00 smithi144 bash[19242]: cluster 2024-06-30T13:10:00.397952+0000 mgr.y (mgr.14144) 241 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:02.918 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:02 smithi167 systemd[1]: Started Ceph grafana.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:10:03.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:03 smithi144 bash[14758]: cluster 2024-06-30T13:10:02.398434+0000 mgr.y (mgr.14144) 242 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:03.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:03 smithi144 bash[14758]: audit 2024-06-30T13:10:02.853475+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:03.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:03 smithi144 bash[14758]: audit 2024-06-30T13:10:02.855476+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:10:03.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:03 smithi144 bash[19242]: cluster 2024-06-30T13:10:02.398434+0000 mgr.y (mgr.14144) 242 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:03.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:03 smithi144 bash[19242]: audit 2024-06-30T13:10:02.853475+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:03.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:03 smithi144 bash[19242]: audit 2024-06-30T13:10:02.855476+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:10:03.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:03 smithi167 bash[14619]: cluster 2024-06-30T13:10:02.398434+0000 mgr.y (mgr.14144) 242 : cluster [DBG] pgmap v223: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:03.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:03 smithi167 bash[14619]: audit 2024-06-30T13:10:02.853475+0000 mon.a (mon.0) 488 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:03.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:03 smithi167 bash[14619]: audit 2024-06-30T13:10:02.855476+0000 mon.a (mon.0) 489 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:10:04.194 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Starting Grafana" logger=server version=6.7.4 commit=8e44bbc5f5 branch=HEAD compiled=2020-05-26T17:35:38+0000 2024-06-30T13:10:04.194 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2024-06-30T13:10:04.194 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2024-06-30T13:10:04.194 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2024-06-30T13:10:04.194 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="App mode production" logger=settings 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Initializing SqlStore" logger=server 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Starting DB migration" logger=migrator 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="create migration_log table" 2024-06-30T13:10:04.195 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="create user table" 2024-06-30T13:10:04.196 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.login" 2024-06-30T13:10:04.196 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.email" 2024-06-30T13:10:04.496 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_login - v1" 2024-06-30T13:10:04.496 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_email - v1" 2024-06-30T13:10:04.496 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table user to user_v1 - v1" 2024-06-30T13:10:04.496 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="create user table v2" 2024-06-30T13:10:04.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_login - v2" 2024-06-30T13:10:04.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_email - v2" 2024-06-30T13:10:04.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-06-30T13:10:04.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table user_v1" 2024-06-30T13:10:04.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="Add column help_flags1 to user table" 2024-06-30T13:10:05.100 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="Update user table charset" 2024-06-30T13:10:05.100 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="Add last_seen_at column to user" 2024-06-30T13:10:05.100 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="Add missing user data" 2024-06-30T13:10:05.100 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:04 smithi167 bash[28664]: t=2024-06-30T13:10:04+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_disabled column to user" 2024-06-30T13:10:05.100 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create temp user table v1-7" 2024-06-30T13:10:05.100 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v1-7" 2024-06-30T13:10:05.352 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v1-7" 2024-06-30T13:10:05.352 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v1-7" 2024-06-30T13:10:05.352 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v1-7" 2024-06-30T13:10:05.352 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="Update temp_user table charset" 2024-06-30T13:10:05.656 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create star table" 2024-06-30T13:10:05.656 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index star.user_id_dashboard_id" 2024-06-30T13:10:05.656 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create org table v1" 2024-06-30T13:10:05.656 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_name - v1" 2024-06-30T13:10:05.656 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create org_user table v1" 2024-06-30T13:10:05.656 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[14619]: cluster 2024-06-30T13:10:04.399049+0000 mgr.y (mgr.14144) 243 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:05 smithi144 bash[14758]: cluster 2024-06-30T13:10:04.399049+0000 mgr.y (mgr.14144) 243 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:05.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:05 smithi144 bash[19242]: cluster 2024-06-30T13:10:04.399049+0000 mgr.y (mgr.14144) 243 : cluster [DBG] pgmap v224: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:05.947 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_org_id - v1" 2024-06-30T13:10:05.948 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_user_org_id_user_id - v1" 2024-06-30T13:10:05.948 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="Update org table charset" 2024-06-30T13:10:06.208 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:05 smithi167 bash[28664]: t=2024-06-30T13:10:05+0000 lvl=info msg="Executing migration" logger=migrator id="Update org_user table charset" 2024-06-30T13:10:06.208 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate all Read Only Viewers to Viewers" 2024-06-30T13:10:06.209 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard table" 2024-06-30T13:10:06.209 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard.account_id" 2024-06-30T13:10:06.209 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_account_id_slug" 2024-06-30T13:10:06.477 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_tag table" 2024-06-30T13:10:06.477 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_tag.dasboard_id_term" 2024-06-30T13:10:06.477 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-06-30T13:10:06.562 INFO:teuthology.orchestra.run.smithi144.stdout:55834574899 2024-06-30T13:10:06.562 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-30T13:10:06.563 INFO:teuthology.orchestra.run.smithi144.stdout:154618822687 2024-06-30T13:10:06.564 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-30T13:10:06.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:06 smithi144 bash[14758]: cluster 2024-06-30T13:10:06.399593+0000 mgr.y (mgr.14144) 244 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:06.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:06 smithi144 bash[19242]: cluster 2024-06-30T13:10:06.399593+0000 mgr.y (mgr.14144) 244 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:06.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[14619]: cluster 2024-06-30T13:10:06.399593+0000 mgr.y (mgr.14144) 244 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:06.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard to dashboard_v1 - v1" 2024-06-30T13:10:06.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard v2" 2024-06-30T13:10:06.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_org_id - v2" 2024-06-30T13:10:07.106 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_org_id_slug - v2" 2024-06-30T13:10:07.106 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard v1 to v2" 2024-06-30T13:10:07.106 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_v1" 2024-06-30T13:10:07.106 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:06 smithi167 bash[28664]: t=2024-06-30T13:10:06+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard.data to mediumtext v1" 2024-06-30T13:10:07.106 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column updated_by in dashboard - v2" 2024-06-30T13:10:07.357 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column created_by in dashboard - v2" 2024-06-30T13:10:07.357 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column gnetId in dashboard" 2024-06-30T13:10:07.358 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for gnetId in dashboard" 2024-06-30T13:10:07.358 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_id in dashboard" 2024-06-30T13:10:07.358 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for plugin_id in dashboard" 2024-06-30T13:10:07.358 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_id in dashboard_tag" 2024-06-30T13:10:07.621 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard table charset" 2024-06-30T13:10:07.621 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_tag table charset" 2024-06-30T13:10:07.622 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column folder_id in dashboard" 2024-06-30T13:10:07.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:07 smithi144 bash[14758]: audit 2024-06-30T13:10:06.617515+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:07.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:07 smithi144 bash[19242]: audit 2024-06-30T13:10:06.617515+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:07.944 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[14619]: audit 2024-06-30T13:10:06.617515+0000 mon.a (mon.0) 490 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:07.944 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column isFolder in dashboard" 2024-06-30T13:10:07.945 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column has_acl in dashboard" 2024-06-30T13:10:07.945 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in dashboard" 2024-06-30T13:10:07.945 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in dashboard" 2024-06-30T13:10:08.088 INFO:teuthology.orchestra.run.smithi144.stdout:77309411373 2024-06-30T13:10:08.088 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-30T13:10:08.204 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:07 smithi167 bash[28664]: t=2024-06-30T13:10:07+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index dashboard_org_id_uid" 2024-06-30T13:10:08.204 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_slug" 2024-06-30T13:10:08.204 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard title length" 2024-06-30T13:10:08.204 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index for dashboard_org_id_title_folder_id" 2024-06-30T13:10:08.204 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning" 2024-06-30T13:10:08.489 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-06-30T13:10:08.489 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning v2" 2024-06-30T13:10:08.489 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-06-30T13:10:08.766 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[14619]: cluster 2024-06-30T13:10:08.400234+0000 mgr.y (mgr.14144) 245 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:08.767 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-06-30T13:10:08.767 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard_provisioning v1 to v2" 2024-06-30T13:10:08.767 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="drop dashboard_provisioning_tmp_qwerty" 2024-06-30T13:10:08.767 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="Add check_sum column" 2024-06-30T13:10:08.767 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table" 2024-06-30T13:10:08.767 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="add index data_source.account_id" 2024-06-30T13:10:08.792 INFO:teuthology.orchestra.run.smithi144.stdout:107374182439 2024-06-30T13:10:08.793 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-30T13:10:09.002 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:08 smithi144 bash[19242]: cluster 2024-06-30T13:10:08.400234+0000 mgr.y (mgr.14144) 245 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:09.003 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:08 smithi144 bash[14758]: cluster 2024-06-30T13:10:08.400234+0000 mgr.y (mgr.14144) 245 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:09.003 INFO:teuthology.orchestra.run.smithi144.stdout:34359738426 2024-06-30T13:10:09.004 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-30T13:10:09.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index data_source.account_id_name" 2024-06-30T13:10:09.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:08 smithi167 bash[28664]: t=2024-06-30T13:10:08+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_data_source_account_id - v1" 2024-06-30T13:10:09.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_data_source_account_id_name - v1" 2024-06-30T13:10:09.196 INFO:teuthology.orchestra.run.smithi144.stdout:128849018916 2024-06-30T13:10:09.196 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-30T13:10:09.345 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table data_source to data_source_v1 - v1" 2024-06-30T13:10:09.345 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table v2" 2024-06-30T13:10:09.345 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_data_source_org_id - v2" 2024-06-30T13:10:09.345 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_data_source_org_id_name - v2" 2024-06-30T13:10:09.345 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-06-30T13:10:09.346 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table data_source_v1 #2" 2024-06-30T13:10:09.613 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="Add column with_credentials" 2024-06-30T13:10:09.613 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="Add secure json data column" 2024-06-30T13:10:09.613 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="Update data_source table charset" 2024-06-30T13:10:09.873 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="Update initial version to 1" 2024-06-30T13:10:09.873 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="Add read_only data column" 2024-06-30T13:10:09.873 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate logging ds to loki ds" 2024-06-30T13:10:09.873 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="Update json_data with nulls" 2024-06-30T13:10:09.874 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table" 2024-06-30T13:10:09.965 INFO:teuthology.orchestra.run.smithi144.stdout:206158430229 2024-06-30T13:10:10.019 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-30T13:10:10.060 INFO:teuthology.orchestra.run.smithi144.stdout:180388626458 2024-06-30T13:10:10.060 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-30T13:10:10.141 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:09 smithi167 bash[28664]: t=2024-06-30T13:10:09+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id" 2024-06-30T13:10:10.142 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.key" 2024-06-30T13:10:10.142 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id_name" 2024-06-30T13:10:10.142 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_api_key_account_id - v1" 2024-06-30T13:10:10.410 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_key - v1" 2024-06-30T13:10:10.411 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_account_id_name - v1" 2024-06-30T13:10:10.411 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table api_key to api_key_v1 - v1" 2024-06-30T13:10:10.411 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table v2" 2024-06-30T13:10:10.411 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_api_key_org_id - v2" 2024-06-30T13:10:10.754 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_key - v2" 2024-06-30T13:10:10.754 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_org_id_name - v2" 2024-06-30T13:10:10.754 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="copy api_key v1 to v2" 2024-06-30T13:10:10.754 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table api_key_v1" 2024-06-30T13:10:10.755 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="Update api_key table charset" 2024-06-30T13:10:10.755 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="Add expires to api_key table" 2024-06-30T13:10:11.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2024-06-30T13:10:11.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2024-06-30T13:10:11.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:10 smithi167 bash[28664]: t=2024-06-30T13:10:10+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2024-06-30T13:10:11.374 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2024-06-30T13:10:11.374 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-06-30T13:10:11.374 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2024-06-30T13:10:11.374 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2024-06-30T13:10:11.374 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2024-06-30T13:10:11.517 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:11.521 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:11.522 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:11.522 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:11.523 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:11.525 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:11.626 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[14619]: cluster 2024-06-30T13:10:10.400869+0000 mgr.y (mgr.14144) 246 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:11.626 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[14619]: audit 2024-06-30T13:10:11.382865+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:11.626 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[14619]: audit 2024-06-30T13:10:11.391294+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:11.627 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2024-06-30T13:10:11.627 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2024-06-30T13:10:11.627 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2024-06-30T13:10:11.627 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2024-06-30T13:10:11.706 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:11 smithi144 bash[14758]: cluster 2024-06-30T13:10:10.400869+0000 mgr.y (mgr.14144) 246 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:11.706 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:11 smithi144 bash[14758]: audit 2024-06-30T13:10:11.382865+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:11.706 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:11 smithi144 bash[14758]: audit 2024-06-30T13:10:11.391294+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:11.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:11 smithi144 bash[19242]: cluster 2024-06-30T13:10:10.400869+0000 mgr.y (mgr.14144) 246 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:11.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:11 smithi144 bash[19242]: audit 2024-06-30T13:10:11.382865+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:11.707 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:11 smithi144 bash[19242]: audit 2024-06-30T13:10:11.391294+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:11.951 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2024-06-30T13:10:11.951 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-06-30T13:10:11.951 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2024-06-30T13:10:11.951 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2024-06-30T13:10:12.255 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:11 smithi167 bash[28664]: t=2024-06-30T13:10:11+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2024-06-30T13:10:12.255 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2024-06-30T13:10:12.255 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2024-06-30T13:10:12.255 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2024-06-30T13:10:12.255 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2024-06-30T13:10:12.255 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2024-06-30T13:10:12.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[14619]: cephadm 2024-06-30T13:10:11.397681+0000 mgr.y (mgr.14144) 247 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-06-30T13:10:12.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[14619]: cephadm 2024-06-30T13:10:11.400937+0000 mgr.y (mgr.14144) 248 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi144 2024-06-30T13:10:12.527 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2024-06-30T13:10:12.527 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2024-06-30T13:10:12.528 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2024-06-30T13:10:12.532 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:12.533 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:12.725 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:12 smithi144 bash[14758]: cephadm 2024-06-30T13:10:11.397681+0000 mgr.y (mgr.14144) 247 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-06-30T13:10:12.725 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:12 smithi144 bash[14758]: cephadm 2024-06-30T13:10:11.400937+0000 mgr.y (mgr.14144) 248 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi144 2024-06-30T13:10:12.725 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:12 smithi144 bash[19242]: cephadm 2024-06-30T13:10:11.397681+0000 mgr.y (mgr.14144) 247 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-06-30T13:10:12.725 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:12 smithi144 bash[19242]: cephadm 2024-06-30T13:10:11.400937+0000 mgr.y (mgr.14144) 248 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi144 2024-06-30T13:10:12.800 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2024-06-30T13:10:12.800 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2024-06-30T13:10:12.800 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2024-06-30T13:10:12.800 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2024-06-30T13:10:13.102 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2024-06-30T13:10:13.102 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2024-06-30T13:10:13.102 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2024-06-30T13:10:13.103 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:12 smithi167 bash[28664]: t=2024-06-30T13:10:12+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2024-06-30T13:10:13.103 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2024-06-30T13:10:13.353 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-06-30T13:10:13.354 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2024-06-30T13:10:13.354 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2024-06-30T13:10:13.354 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2024-06-30T13:10:13.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:13 smithi144 bash[14758]: cluster 2024-06-30T13:10:12.401573+0000 mgr.y (mgr.14144) 249 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:13.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:13 smithi144 bash[19242]: cluster 2024-06-30T13:10:12.401573+0000 mgr.y (mgr.14144) 249 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:13.605 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[14619]: cluster 2024-06-30T13:10:12.401573+0000 mgr.y (mgr.14144) 249 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:13.605 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2024-06-30T13:10:13.605 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2024-06-30T13:10:13.605 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2024-06-30T13:10:13.605 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2024-06-30T13:10:13.605 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2024-06-30T13:10:13.906 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2024-06-30T13:10:13.907 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2024-06-30T13:10:13.907 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2024-06-30T13:10:14.166 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2024-06-30T13:10:14.166 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:13 smithi167 bash[28664]: t=2024-06-30T13:10:13+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-06-30T13:10:14.166 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2024-06-30T13:10:14.166 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2024-06-30T13:10:14.435 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2024-06-30T13:10:14.436 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2024-06-30T13:10:14.436 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2024-06-30T13:10:14.436 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2024-06-30T13:10:14.745 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2024-06-30T13:10:14.745 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2024-06-30T13:10:14.745 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2024-06-30T13:10:14.745 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2024-06-30T13:10:14.746 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2024-06-30T13:10:14.746 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2024-06-30T13:10:15.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2024-06-30T13:10:15.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2024-06-30T13:10:15.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:14 smithi167 bash[28664]: t=2024-06-30T13:10:14+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2024-06-30T13:10:15.306 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2024-06-30T13:10:15.306 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2024-06-30T13:10:15.307 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2024-06-30T13:10:15.307 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2024-06-30T13:10:15.625 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[14619]: cluster 2024-06-30T13:10:14.402191+0000 mgr.y (mgr.14144) 250 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:15.626 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2024-06-30T13:10:15.626 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2024-06-30T13:10:15.626 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2024-06-30T13:10:15.626 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2024-06-30T13:10:15.626 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2024-06-30T13:10:15.626 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2024-06-30T13:10:15.735 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:15 smithi144 bash[19242]: cluster 2024-06-30T13:10:14.402191+0000 mgr.y (mgr.14144) 250 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:15.736 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:15 smithi144 bash[14758]: cluster 2024-06-30T13:10:14.402191+0000 mgr.y (mgr.14144) 250 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:15.893 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2024-06-30T13:10:15.893 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2024-06-30T13:10:15.894 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2024-06-30T13:10:15.894 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2024-06-30T13:10:15.894 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-06-30T13:10:16.187 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:15 smithi167 bash[28664]: t=2024-06-30T13:10:15+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-06-30T13:10:16.187 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2024-06-30T13:10:16.187 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2024-06-30T13:10:16.187 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2024-06-30T13:10:16.472 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2024-06-30T13:10:16.472 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2024-06-30T13:10:16.472 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2024-06-30T13:10:16.473 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-06-30T13:10:16.765 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2024-06-30T13:10:16.766 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2024-06-30T13:10:16.766 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2024-06-30T13:10:16.766 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2024-06-30T13:10:16.766 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2024-06-30T13:10:16.766 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2024-06-30T13:10:17.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2024-06-30T13:10:17.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2024-06-30T13:10:17.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2024-06-30T13:10:17.027 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:16 smithi167 bash[28664]: t=2024-06-30T13:10:16+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2024-06-30T13:10:17.353 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2024-06-30T13:10:17.353 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2024-06-30T13:10:17.353 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2024-06-30T13:10:17.353 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2024-06-30T13:10:17.353 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2024-06-30T13:10:17.354 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2024-06-30T13:10:17.624 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:17 smithi144 bash[14758]: cluster 2024-06-30T13:10:16.402769+0000 mgr.y (mgr.14144) 251 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:17.625 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:17 smithi144 bash[19242]: cluster 2024-06-30T13:10:16.402769+0000 mgr.y (mgr.14144) 251 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:17.654 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[14619]: cluster 2024-06-30T13:10:16.402769+0000 mgr.y (mgr.14144) 251 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:17.655 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2024-06-30T13:10:17.655 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2024-06-30T13:10:17.655 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2024-06-30T13:10:17.655 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2024-06-30T13:10:17.973 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2024-06-30T13:10:17.973 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2024-06-30T13:10:17.974 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-06-30T13:10:17.974 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2024-06-30T13:10:17.974 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:17 smithi167 bash[28664]: t=2024-06-30T13:10:17+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2024-06-30T13:10:18.266 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2024-06-30T13:10:18.267 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2024-06-30T13:10:18.267 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2024-06-30T13:10:18.527 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-06-30T13:10:18.527 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2024-06-30T13:10:18.527 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2024-06-30T13:10:18.527 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2024-06-30T13:10:18.527 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2024-06-30T13:10:18.870 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2024-06-30T13:10:18.871 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2024-06-30T13:10:18.871 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2024-06-30T13:10:18.871 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2024-06-30T13:10:19.139 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2024-06-30T13:10:19.139 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:18 smithi167 bash[28664]: t=2024-06-30T13:10:18+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2024-06-30T13:10:19.139 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2024-06-30T13:10:19.139 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2024-06-30T13:10:19.139 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2024-06-30T13:10:19.417 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Created default admin" logger=sqlstore user=admin 2024-06-30T13:10:19.417 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing HTTPServer" logger=server 2024-06-30T13:10:19.417 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing BackendPluginManager" logger=server 2024-06-30T13:10:19.417 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing PluginManager" logger=server 2024-06-30T13:10:19.417 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Starting plugin search" logger=plugins 2024-06-30T13:10:19.417 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Registering plugin" logger=plugins name="Pie Chart" 2024-06-30T13:10:19.417 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Registering plugin" logger=plugins name="Status Panel" 2024-06-30T13:10:19.417 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing HooksService" logger=server 2024-06-30T13:10:19.417 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing OSSLicensingService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing InternalMetricsService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing RemoteCache" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing RenderingService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing AlertEngine" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing QuotaService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing ServerLockService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing UserAuthTokenService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing DatasourceCacheService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing LoginService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing SearchService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing TracingService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing UsageStatsService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing CleanUpService" logger=server 2024-06-30T13:10:19.418 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing NotificationService" logger=server 2024-06-30T13:10:19.419 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing provisioningServiceImpl" logger=server 2024-06-30T13:10:19.419 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-06-30T13:10:19.419 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 2024-06-30T13:10:19.419 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Backend rendering via phantomJS" logger=rendering renderer=phantomJS 2024-06-30T13:10:19.419 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=warn msg="phantomJS is deprecated and will be removed in a future release. You should consider migrating from phantomJS to grafana-image-renderer plugin. Read more at https://grafana.com/docs/grafana/latest/administration/image_rendering/" logger=rendering renderer=phantomJS 2024-06-30T13:10:19.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:19 smithi144 bash[14758]: cluster 2024-06-30T13:10:18.403358+0000 mgr.y (mgr.14144) 252 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:19.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:19 smithi144 bash[19242]: cluster 2024-06-30T13:10:18.403358+0000 mgr.y (mgr.14144) 252 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:19.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[14619]: cluster 2024-06-30T13:10:18.403358+0000 mgr.y (mgr.14144) 252 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:19.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="Initializing Stream Manager" 2024-06-30T13:10:19.777 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:10:19 smithi167 bash[28664]: t=2024-06-30T13:10:19+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-06-30T13:10:20.482 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:20 smithi144 bash[19242]: cluster 2024-06-30T13:10:20.403911+0000 mgr.y (mgr.14144) 253 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:20.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:20 smithi144 bash[14758]: cluster 2024-06-30T13:10:20.403911+0000 mgr.y (mgr.14144) 253 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:20.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:20 smithi167 bash[14619]: cluster 2024-06-30T13:10:20.403911+0000 mgr.y (mgr.14144) 253 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:23.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:23 smithi144 bash[19242]: cluster 2024-06-30T13:10:22.404453+0000 mgr.y (mgr.14144) 254 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:23.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:23 smithi144 bash[14758]: cluster 2024-06-30T13:10:22.404453+0000 mgr.y (mgr.14144) 254 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:23.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:23 smithi167 bash[14619]: cluster 2024-06-30T13:10:22.404453+0000 mgr.y (mgr.14144) 254 : cluster [DBG] pgmap v233: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:24.991 INFO:teuthology.orchestra.run.smithi144.stdout:34359738430 2024-06-30T13:10:24.999 INFO:teuthology.orchestra.run.smithi144.stdout:55834574903 2024-06-30T13:10:25.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:25 smithi144 bash[14758]: cluster 2024-06-30T13:10:24.405027+0000 mgr.y (mgr.14144) 255 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:25.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:25 smithi144 bash[14758]: audit 2024-06-30T13:10:24.988325+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? 172.21.15.144:0/967114169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-30T13:10:25.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:25 smithi144 bash[14758]: audit 2024-06-30T13:10:24.996998+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.144:0/1780880270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-30T13:10:25.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:25 smithi144 bash[19242]: cluster 2024-06-30T13:10:24.405027+0000 mgr.y (mgr.14144) 255 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:25.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:25 smithi144 bash[19242]: audit 2024-06-30T13:10:24.988325+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? 172.21.15.144:0/967114169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-30T13:10:25.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:25 smithi144 bash[19242]: audit 2024-06-30T13:10:24.996998+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.144:0/1780880270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-30T13:10:25.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:25 smithi167 bash[14619]: cluster 2024-06-30T13:10:24.405027+0000 mgr.y (mgr.14144) 255 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:25.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:25 smithi167 bash[14619]: audit 2024-06-30T13:10:24.988325+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? 172.21.15.144:0/967114169' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-30T13:10:25.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:25 smithi167 bash[14619]: audit 2024-06-30T13:10:24.996998+0000 mon.c (mon.2) 12 : audit [DBG] from='client.? 172.21.15.144:0/1780880270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-30T13:10:26.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:26 smithi144 bash[19242]: cluster 2024-06-30T13:10:26.405604+0000 mgr.y (mgr.14144) 256 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:26.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:26 smithi144 bash[14758]: cluster 2024-06-30T13:10:26.405604+0000 mgr.y (mgr.14144) 256 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:26.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:26 smithi167 bash[14619]: cluster 2024-06-30T13:10:26.405604+0000 mgr.y (mgr.14144) 256 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:27.000 INFO:teuthology.orchestra.run.smithi144.stdout:154618822691 2024-06-30T13:10:27.127 INFO:teuthology.orchestra.run.smithi144.stdout:128849018920 2024-06-30T13:10:27.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:27 smithi144 bash[14758]: audit 2024-06-30T13:10:26.997800+0000 mon.a (mon.0) 493 : audit [DBG] from='client.? 172.21.15.144:0/2211119302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-30T13:10:27.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:27 smithi144 bash[14758]: audit 2024-06-30T13:10:27.125105+0000 mon.a (mon.0) 494 : audit [DBG] from='client.? 172.21.15.144:0/2016444696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-30T13:10:27.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:27 smithi144 bash[19242]: audit 2024-06-30T13:10:26.997800+0000 mon.a (mon.0) 493 : audit [DBG] from='client.? 172.21.15.144:0/2211119302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-30T13:10:27.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:27 smithi144 bash[19242]: audit 2024-06-30T13:10:27.125105+0000 mon.a (mon.0) 494 : audit [DBG] from='client.? 172.21.15.144:0/2016444696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-30T13:10:27.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:27 smithi167 bash[14619]: audit 2024-06-30T13:10:26.997800+0000 mon.a (mon.0) 493 : audit [DBG] from='client.? 172.21.15.144:0/2211119302' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-30T13:10:27.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:27 smithi167 bash[14619]: audit 2024-06-30T13:10:27.125105+0000 mon.a (mon.0) 494 : audit [DBG] from='client.? 172.21.15.144:0/2016444696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-30T13:10:28.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574899 got 55834574903 for osd.1 2024-06-30T13:10:28.688 DEBUG:teuthology.parallel:result is None 2024-06-30T13:10:28.689 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738426 got 34359738430 for osd.0 2024-06-30T13:10:28.689 DEBUG:teuthology.parallel:result is None 2024-06-30T13:10:28.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:28 smithi144 bash[14758]: cluster 2024-06-30T13:10:28.406175+0000 mgr.y (mgr.14144) 257 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:28.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:28 smithi144 bash[19242]: cluster 2024-06-30T13:10:28.406175+0000 mgr.y (mgr.14144) 257 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:28.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:28 smithi167 bash[14619]: cluster 2024-06-30T13:10:28.406175+0000 mgr.y (mgr.14144) 257 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:29.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822687 got 154618822691 for osd.5 2024-06-30T13:10:29.878 DEBUG:teuthology.parallel:result is None 2024-06-30T13:10:29.916 INFO:teuthology.orchestra.run.smithi144.stdout:77309411377 2024-06-30T13:10:30.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:30 smithi144 bash[14758]: audit 2024-06-30T13:10:29.914279+0000 mon.c (mon.2) 13 : audit [DBG] from='client.? 172.21.15.144:0/4088904622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-30T13:10:30.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:30 smithi144 bash[19242]: audit 2024-06-30T13:10:29.914279+0000 mon.c (mon.2) 13 : audit [DBG] from='client.? 172.21.15.144:0/4088904622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-30T13:10:30.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:30 smithi167 bash[14619]: audit 2024-06-30T13:10:29.914279+0000 mon.c (mon.2) 13 : audit [DBG] from='client.? 172.21.15.144:0/4088904622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-30T13:10:31.272 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018916 got 128849018920 for osd.4 2024-06-30T13:10:31.273 DEBUG:teuthology.parallel:result is None 2024-06-30T13:10:31.433 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411373 got 77309411377 for osd.2 2024-06-30T13:10:31.433 DEBUG:teuthology.parallel:result is None 2024-06-30T13:10:31.730 INFO:teuthology.orchestra.run.smithi144.stdout:180388626463 2024-06-30T13:10:31.745 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:31 smithi144 bash[14758]: cluster 2024-06-30T13:10:30.406757+0000 mgr.y (mgr.14144) 258 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:31.746 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:31 smithi144 bash[19242]: cluster 2024-06-30T13:10:30.406757+0000 mgr.y (mgr.14144) 258 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:31.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:31 smithi167 bash[14619]: cluster 2024-06-30T13:10:30.406757+0000 mgr.y (mgr.14144) 258 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:31.797 INFO:teuthology.orchestra.run.smithi144.stdout:206158430234 2024-06-30T13:10:31.840 INFO:teuthology.orchestra.run.smithi144.stdout:107374182444 2024-06-30T13:10:32.757 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:32 smithi144 bash[14758]: audit 2024-06-30T13:10:31.728474+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? 172.21.15.144:0/1834807994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-30T13:10:32.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:32 smithi144 bash[14758]: audit 2024-06-30T13:10:31.794941+0000 mon.a (mon.0) 495 : audit [DBG] from='client.? 172.21.15.144:0/433868309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-30T13:10:32.758 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:32 smithi144 bash[14758]: audit 2024-06-30T13:10:31.837810+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.144:0/2602529661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-30T13:10:32.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:32 smithi144 bash[19242]: audit 2024-06-30T13:10:31.728474+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? 172.21.15.144:0/1834807994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-30T13:10:32.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:32 smithi144 bash[19242]: audit 2024-06-30T13:10:31.794941+0000 mon.a (mon.0) 495 : audit [DBG] from='client.? 172.21.15.144:0/433868309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-30T13:10:32.758 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:32 smithi144 bash[19242]: audit 2024-06-30T13:10:31.837810+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.144:0/2602529661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-30T13:10:32.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:32 smithi167 bash[14619]: audit 2024-06-30T13:10:31.728474+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? 172.21.15.144:0/1834807994' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-30T13:10:32.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:32 smithi167 bash[14619]: audit 2024-06-30T13:10:31.794941+0000 mon.a (mon.0) 495 : audit [DBG] from='client.? 172.21.15.144:0/433868309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-30T13:10:32.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:32 smithi167 bash[14619]: audit 2024-06-30T13:10:31.837810+0000 mon.c (mon.2) 15 : audit [DBG] from='client.? 172.21.15.144:0/2602529661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-30T13:10:33.009 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:32 smithi144 systemd[1]: Stopping Ceph alertmanager.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:10:33.762 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:33 smithi144 bash[32518]: level=info ts=2024-06-30T13:10:33.336Z caller=main.go:536 msg="Received SIGTERM, exiting gracefully..." 2024-06-30T13:10:33.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:33 smithi144 bash[14758]: cluster 2024-06-30T13:10:32.407273+0000 mgr.y (mgr.14144) 259 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:33.763 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:33 smithi144 bash[19242]: cluster 2024-06-30T13:10:32.407273+0000 mgr.y (mgr.14144) 259 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:33.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:33 smithi167 bash[14619]: cluster 2024-06-30T13:10:32.407273+0000 mgr.y (mgr.14144) 259 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:33.883 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626458 got 180388626463 for osd.6 2024-06-30T13:10:33.883 DEBUG:teuthology.parallel:result is None 2024-06-30T13:10:34.509 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:34 smithi144 docker[36569]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-alertmanager.a 2024-06-30T13:10:34.628 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182439 got 107374182444 for osd.3 2024-06-30T13:10:34.629 DEBUG:teuthology.parallel:result is None 2024-06-30T13:10:34.631 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430229 got 206158430234 for osd.7 2024-06-30T13:10:34.631 DEBUG:teuthology.parallel:result is None 2024-06-30T13:10:34.631 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-30T13:10:34.631 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph pg dump --format=json 2024-06-30T13:10:35.009 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:34 smithi144 systemd[1]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@alertmanager.a.service: Succeeded. 2024-06-30T13:10:35.010 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:34 smithi144 systemd[1]: Stopped Ceph alertmanager.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:10:35.010 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:34 smithi144 systemd[1]: Started Ceph alertmanager.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:10:35.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[14758]: cluster 2024-06-30T13:10:34.407861+0000 mgr.y (mgr.14144) 260 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:35.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[14758]: audit 2024-06-30T13:10:34.695717+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:35.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[19242]: cluster 2024-06-30T13:10:34.407861+0000 mgr.y (mgr.14144) 260 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:35.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[19242]: audit 2024-06-30T13:10:34.695717+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:35 smithi167 bash[14619]: cluster 2024-06-30T13:10:34.407861+0000 mgr.y (mgr.14144) 260 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:35 smithi167 bash[14619]: audit 2024-06-30T13:10:34.695717+0000 mon.a (mon.0) 496 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:36.259 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[36617]: level=info ts=2024-06-30T13:10:35.929Z caller=main.go:231 msg="Starting Alertmanager" version="(version=0.20.0, branch=HEAD, revision=f74be0400a6243d10bb53812d6fa408ad71ff32d)" 2024-06-30T13:10:36.260 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[36617]: level=info ts=2024-06-30T13:10:35.929Z caller=main.go:232 build_context="(go=go1.13.5, user=root@00c3106655f8, date=20191211-14:13:14)" 2024-06-30T13:10:36.260 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[36617]: level=info ts=2024-06-30T13:10:35.931Z caller=cluster.go:161 component=cluster msg="setting advertise address explicitly" addr=172.21.15.144 port=9094 2024-06-30T13:10:36.260 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[36617]: level=info ts=2024-06-30T13:10:35.932Z caller=cluster.go:623 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-30T13:10:36.260 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[36617]: level=info ts=2024-06-30T13:10:35.971Z caller=coordinator.go:119 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-30T13:10:36.260 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[36617]: level=info ts=2024-06-30T13:10:35.971Z caller=coordinator.go:131 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-30T13:10:36.260 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:35 smithi144 bash[36617]: level=info ts=2024-06-30T13:10:35.975Z caller=main.go:497 msg=Listening address=:9093 2024-06-30T13:10:36.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:36 smithi144 bash[14758]: cephadm 2024-06-30T13:10:34.697227+0000 mgr.y (mgr.14144) 261 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-06-30T13:10:36.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:36 smithi144 bash[14758]: cephadm 2024-06-30T13:10:34.701143+0000 mgr.y (mgr.14144) 262 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi167 2024-06-30T13:10:36.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:36 smithi144 bash[14758]: cluster 2024-06-30T13:10:36.408482+0000 mgr.y (mgr.14144) 263 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:36.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:36 smithi144 bash[19242]: cephadm 2024-06-30T13:10:34.697227+0000 mgr.y (mgr.14144) 261 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-06-30T13:10:36.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:36 smithi144 bash[19242]: cephadm 2024-06-30T13:10:34.701143+0000 mgr.y (mgr.14144) 262 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi167 2024-06-30T13:10:36.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:36 smithi144 bash[19242]: cluster 2024-06-30T13:10:36.408482+0000 mgr.y (mgr.14144) 263 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:36.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:36 smithi167 bash[14619]: cephadm 2024-06-30T13:10:34.697227+0000 mgr.y (mgr.14144) 261 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-06-30T13:10:36.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:36 smithi167 bash[14619]: cephadm 2024-06-30T13:10:34.701143+0000 mgr.y (mgr.14144) 262 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi167 2024-06-30T13:10:36.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:36 smithi167 bash[14619]: cluster 2024-06-30T13:10:36.408482+0000 mgr.y (mgr.14144) 263 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:38.259 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:37 smithi144 bash[36617]: level=info ts=2024-06-30T13:10:37.933Z caller=cluster.go:648 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000205604s 2024-06-30T13:10:38.363 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:38.947 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 systemd[1]: Stopping Ceph prometheus.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:10:38.947 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=warn ts=2024-06-30T13:10:38.765Z caller=main.go:524 msg="Received SIGTERM, exiting gracefully..." 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=main.go:547 msg="Stopping scrape discovery manager..." 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=main.go:561 msg="Stopping notify discovery manager..." 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=main.go:583 msg="Stopping scrape manager..." 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=main.go:543 msg="Scrape discovery manager stopped" 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=manager.go:882 component="rule manager" msg="Stopping rule manager..." 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=manager.go:892 component="rule manager" msg="Rule manager stopped" 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=main.go:577 msg="Scrape manager stopped" 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=main.go:557 msg="Notify discovery manager stopped" 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=notifier.go:601 component=notifier msg="Stopping notification manager..." 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.765Z caller=main.go:749 msg="Notifier manager stopped" 2024-06-30T13:10:38.948 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:38 smithi167 bash[27779]: level=info ts=2024-06-30T13:10:38.766Z caller=main.go:761 msg="See you next time!" 2024-06-30T13:10:39.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:39 smithi167 bash[14619]: cluster 2024-06-30T13:10:38.409073+0000 mgr.y (mgr.14144) 264 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:39.527 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:39 smithi167 docker[29233]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-prometheus.a 2024-06-30T13:10:39.527 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:39 smithi167 systemd[1]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@prometheus.a.service: Succeeded. 2024-06-30T13:10:39.527 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:39 smithi167 systemd[1]: Stopped Ceph prometheus.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:10:39.528 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:39 smithi167 systemd[1]: Started Ceph prometheus.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:10:39.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:39 smithi144 bash[14758]: cluster 2024-06-30T13:10:38.409073+0000 mgr.y (mgr.14144) 264 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:39.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:39 smithi144 bash[19242]: cluster 2024-06-30T13:10:38.409073+0000 mgr.y (mgr.14144) 264 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:40.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.527787+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:40.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.534185+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-30T13:10:40.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.534919+0000 mgr.y (mgr.14144) 265 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-30T13:10:40.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.536579+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144:9093"}]: dispatch 2024-06-30T13:10:40.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.537145+0000 mgr.y (mgr.14144) 266 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144:9093"}]: dispatch 2024-06-30T13:10:40.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.545363+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.550426+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.550925+0000 mgr.y (mgr.14144) 267 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.559224+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi167:9095"}]: dispatch 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.559729+0000 mgr.y (mgr.14144) 268 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi167:9095"}]: dispatch 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.573584+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.580236+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.580834+0000 mgr.y (mgr.14144) 269 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.586778+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi167:3000"}]: dispatch 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.587186+0000 mgr.y (mgr.14144) 270 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi167:3000"}]: dispatch 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.596444+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: audit 2024-06-30T13:10:39.600319+0000 mon.a (mon.0) 507 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:10:40.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[14758]: cluster 2024-06-30T13:10:40.409687+0000 mgr.y (mgr.14144) 271 : cluster [DBG] pgmap v242: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.527787+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.534185+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.534919+0000 mgr.y (mgr.14144) 265 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.536579+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144:9093"}]: dispatch 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.537145+0000 mgr.y (mgr.14144) 266 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144:9093"}]: dispatch 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.545363+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.550426+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.550925+0000 mgr.y (mgr.14144) 267 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.559224+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi167:9095"}]: dispatch 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.559729+0000 mgr.y (mgr.14144) 268 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi167:9095"}]: dispatch 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.573584+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.580236+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-30T13:10:40.762 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.580834+0000 mgr.y (mgr.14144) 269 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-30T13:10:40.763 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.586778+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi167:3000"}]: dispatch 2024-06-30T13:10:40.763 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.587186+0000 mgr.y (mgr.14144) 270 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi167:3000"}]: dispatch 2024-06-30T13:10:40.763 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.596444+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:40.763 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: audit 2024-06-30T13:10:39.600319+0000 mon.a (mon.0) 507 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:10:40.763 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:40 smithi144 bash[19242]: cluster 2024-06-30T13:10:40.409687+0000 mgr.y (mgr.14144) 271 : cluster [DBG] pgmap v242: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:41.027 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.712Z caller=main.go:302 msg="No time or size retention was set so using the default time retention" duration=15d 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.712Z caller=main.go:337 msg="Starting Prometheus" version="(version=2.18.1, branch=HEAD, revision=ecee9c8abfd118f139014cb1b174b08db3f342cf)" 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.712Z caller=main.go:338 build_context="(go=go1.14.2, user=root@2117a9e64a7e, date=20200507-16:51:47)" 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.712Z caller=main.go:339 host_details="(Linux 5.4.0-187-generic #207-Ubuntu SMP Mon Jun 10 08:16:10 UTC 2024 x86_64 smithi167 (none))" 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.712Z caller=main.go:340 fd_limits="(soft=1048576, hard=1048576)" 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.712Z caller=main.go:341 vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.714Z caller=web.go:523 component=web msg="Start listening for connections" address=:9095 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.714Z caller=main.go:678 msg="Starting TSDB ..." 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.722Z caller=head.go:575 component=tsdb msg="Replaying WAL, this may take awhile" 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.723Z caller=head.go:624 component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.723Z caller=head.go:624 component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.723Z caller=head.go:627 component=tsdb msg="WAL replay completed" duration=1.02207ms 2024-06-30T13:10:41.028 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.726Z caller=main.go:694 fs_type=XFS_SUPER_MAGIC 2024-06-30T13:10:41.029 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.726Z caller=main.go:695 msg="TSDB started" 2024-06-30T13:10:41.029 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.726Z caller=main.go:799 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-30T13:10:41.029 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.727Z caller=main.go:827 msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-30T13:10:41.029 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[29292]: level=info ts=2024-06-30T13:10:40.727Z caller=main.go:646 msg="Server is ready to receive web requests." 2024-06-30T13:10:41.029 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.527787+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:41.029 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.534185+0000 mon.a (mon.0) 498 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-30T13:10:41.029 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.534919+0000 mgr.y (mgr.14144) 265 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-30T13:10:41.029 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.536579+0000 mon.a (mon.0) 499 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144:9093"}]: dispatch 2024-06-30T13:10:41.029 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.537145+0000 mgr.y (mgr.14144) 266 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144:9093"}]: dispatch 2024-06-30T13:10:41.029 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.545363+0000 mon.a (mon.0) 500 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:41.029 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.550426+0000 mon.a (mon.0) 501 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.550925+0000 mgr.y (mgr.14144) 267 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.559224+0000 mon.a (mon.0) 502 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi167:9095"}]: dispatch 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.559729+0000 mgr.y (mgr.14144) 268 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi167:9095"}]: dispatch 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.573584+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.580236+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.580834+0000 mgr.y (mgr.14144) 269 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.586778+0000 mon.a (mon.0) 505 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi167:3000"}]: dispatch 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.587186+0000 mgr.y (mgr.14144) 270 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi167:3000"}]: dispatch 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.596444+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: audit 2024-06-30T13:10:39.600319+0000 mon.a (mon.0) 507 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:10:41.030 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:40 smithi167 bash[14619]: cluster 2024-06-30T13:10:40.409687+0000 mgr.y (mgr.14144) 271 : cluster [DBG] pgmap v242: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:41.841 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:10:41.843 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2024-06-30T13:10:42.563 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-06-30T13:10:40.409371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":43220,"kb_used_data":3732,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":749917996,"statfs":{"total":767960285184,"available":767916027904,"internally_reserved":0,"allocated":3821568,"data_stored":1511015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"50","state":"active+clean","last_fresh":"2024-06-30T13:08:39.496069+0000","last_change":"2024-06-30T13:08:39.496069+0000","last_active":"2024-06-30T13:08:39.496069+0000","last_peered":"2024-06-30T13:08:39.496069+0000","last_clean":"2024-06-30T13:08:39.496069+0000","last_became_active":"2024-06-30T13:08:39.495030+0000","last_became_peered":"2024-06-30T13:08:39.495030+0000","last_unstale":"2024-06-30T13:08:39.496069+0000","last_undegraded":"2024-06-30T13:08:39.496069+0000","last_fullsized":"2024-06-30T13:08:39.496069+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-30T13:06:29.438624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-30T13:06:29.438624+0000","last_clean_scrub_stamp":"2024-06-30T13:06:29.438624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430236,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5212,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.036},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.98199999999999998}]},{"osd":1,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":1.258}]},{"osd":2,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.133},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.76300000000000001}]},{"osd":3,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.85099999999999998}]},{"osd":4,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.403},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.091}]},{"osd":5,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":1.3160000000000001}]},{"osd":6,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.4550000000000001}]}]},{"osd":6,"up_from":42,"seq":180388626465,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5276,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":93739876,"statfs":{"total":95995035648,"available":95989633024,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.91100000000000003,"15min":0.90000000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.099},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93200000000000005,"15min":0.93700000000000006},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":1.0569999999999999}]},{"osd":1,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.02,"15min":1.0109999999999999},"min":{"1min":0.74099999999999999,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.92300000000000004,"15min":0.91800000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.177}]},{"osd":2,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95599999999999996,"15min":0.94999999999999996},"min":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.04,"5min":0.996,"15min":0.98799999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.21}]},{"osd":3,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.80700000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.0109999999999999}]},{"osd":4,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.96399999999999997,"15min":0.95999999999999996},"min":{"1min":0.59299999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.052,"5min":1.042,"15min":1.04},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.2729999999999999}]},{"osd":5,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.022,"15min":1.016},"min":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.0980000000000001,"15min":1.089},"min":{"1min":0.80300000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.357}]},{"osd":7,"last update":"Sun Jun 30 13:09:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.137},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.409}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5532,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739620,"statfs":{"total":95995035648,"available":95989370880,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65300000000000002,"15min":0.60599999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.022,"5min":1.024,"15min":1.024},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72599999999999998,"15min":0.64000000000000001},"min":{"1min":0.59499999999999997,"5min":0.441,"15min":0.441},"max":{"1min":1.139,"5min":1.222,"15min":1.222},"last":0.58499999999999996}]},{"osd":2,"last update":"Sun Jun 30 13:09:32 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68000000000000005,"15min":0.65800000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.62},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.79600000000000004,"15min":0.75600000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.96599999999999997}]},{"osd":3,"last update":"Sun Jun 30 13:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.85999999999999999,"15min":0.85199999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.98099999999999998}]},{"osd":4,"last update":"Sun Jun 30 13:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.052,"15min":1.0349999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.2549999999999999,"5min":1.554,"15min":1.554},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.026,"15min":1.026},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.2789999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.345}]},{"osd":5,"last update":"Sun Jun 30 13:09:49 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.073,"15min":1.077},"min":{"1min":0.77500000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.101},{"interface":"front","average":{"1min":1.165,"5min":1.1339999999999999,"15min":1.129},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.2170000000000001}]},{"osd":6,"last update":"Sun Jun 30 13:10:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1399999999999999,"15min":1.1519999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.3400000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1439999999999999,"15min":1.1379999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.1579999999999999}]},{"osd":7,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":1.3120000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738433,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5596,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739556,"statfs":{"total":95995035648,"available":95989305344,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jun 30 13:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65600000000000003,"15min":0.61299999999999999},"min":{"1min":0.441,"5min":0.435,"15min":0.435},"max":{"1min":0.85499999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.72099999999999997,"15min":0.65300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.117,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Jun 30 13:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.79600000000000004,"15min":0.751},"min":{"1min":0.624,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79900000000000004,"15min":0.75900000000000001},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.71799999999999997}]},{"osd":3,"last update":"Sun Jun 30 13:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2350000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.89700000000000002,"15min":0.88300000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.88200000000000001}]},{"osd":4,"last update":"Sun Jun 30 13:10:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0029999999999999,"15min":0.98899999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.127},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0409999999999999,"15min":1.0309999999999999},"min":{"1min":0.77600000000000002,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.0800000000000001}]},{"osd":5,"last update":"Sun Jun 30 13:09:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0309999999999999,"15min":1.0209999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.1379999999999999,"15min":1.1359999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.2490000000000001}]},{"osd":6,"last update":"Sun Jun 30 13:10:17 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.258,"15min":1.2709999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.474,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.385},{"interface":"front","average":{"1min":1.147,"5min":1.234,"15min":1.248},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.425,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.0349999999999999}]},{"osd":7,"last update":"Sun Jun 30 13:09:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.331}]}]},{"osd":2,"up_from":18,"seq":77309411380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5464,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739688,"statfs":{"total":95995035648,"available":95989440512,"internally_reserved":0,"allocated":475136,"data_stored":188680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.71399999999999997,"15min":0.65600000000000003},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.77100000000000002},"min":{"1min":0.52100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.76600000000000001}]},{"osd":1,"last update":"Sun Jun 30 13:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.65700000000000003},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93899999999999995,"5min":1.153,"15min":1.153},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.79500000000000004,"15min":0.73299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1319999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":1.0649999999999999}]},{"osd":3,"last update":"Sun Jun 30 13:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.85099999999999998,"15min":0.82299999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.82899999999999996,"15min":0.83399999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.464,"15min":1.464},"last":0.92700000000000005}]},{"osd":4,"last update":"Sun Jun 30 13:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90600000000000003,"15min":0.88700000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.081,"5min":1.667,"15min":1.667},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.073,"15min":1.0369999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4059999999999999,"5min":1.452,"15min":1.452},"last":1.306}]},{"osd":5,"last update":"Sun Jun 30 13:09:47 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0640000000000001,"15min":1.0680000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.1060000000000001,"15min":1.0860000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":1.4279999999999999}]},{"osd":6,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.228,"15min":1.2270000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.5549999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1439999999999999,"15min":1.1499999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4830000000000001,"5min":1.605,"15min":1.605},"last":1.4870000000000001}]},{"osd":7,"last update":"Sun Jun 30 13:09:42 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.133},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.3819999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5404,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739748,"statfs":{"total":95995035648,"available":95989501952,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.68400000000000005},"min":{"1min":0.56000000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0269999999999999,"5min":1.179,"15min":1.179},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.80800000000000005,"15min":0.77400000000000002},"min":{"1min":0.62,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.27,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.76800000000000002}]},{"osd":1,"last update":"Sun Jun 30 13:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.879},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.96499999999999997}]},{"osd":2,"last update":"Sun Jun 30 13:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.80500000000000005,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1970000000000001,"5min":1.198,"15min":1.198},"last":1.0700000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76300000000000001,"15min":0.73899999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.80800000000000005}]},{"osd":4,"last update":"Sun Jun 30 13:10:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0349999999999999,"15min":1.0229999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.2809999999999999,"5min":1.53,"15min":1.53},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.0960000000000001,"15min":1.0840000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.2929999999999999,"5min":1.552,"15min":1.552},"last":1.248}]},{"osd":5,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0069999999999999,"15min":1.0009999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.087},"min":{"1min":0.88200000000000001,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":1.0249999999999999}]},{"osd":6,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.103,"15min":1.103},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.046,"5min":1.0980000000000001,"15min":1.1060000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.0960000000000001}]},{"osd":7,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.143,"15min":1.143},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.2989999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018923,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5404,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739748,"statfs":{"total":95995035648,"available":95989501952,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96999999999999997,"15min":0.94399999999999995},"min":{"1min":0.72599999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.177},{"interface":"front","average":{"1min":0.999,"5min":1.0289999999999999,"15min":1.006},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.3160000000000001,"5min":1.625,"15min":1.625},"last":1.018}]},{"osd":1,"last update":"Sun Jun 30 13:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.88600000000000001,"15min":0.87},"min":{"1min":0.624,"5min":0.503,"15min":0.503},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.93000000000000005,"15min":0.91600000000000004},"min":{"1min":0.67900000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.214,"5min":1.262,"15min":1.262},"last":0.88100000000000001}]},{"osd":2,"last update":"Sun Jun 30 13:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.96799999999999997,"15min":0.96399999999999997},"min":{"1min":0.69899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.95699999999999996,"15min":0.94299999999999995},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.2350000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":1.1040000000000001}]},{"osd":3,"last update":"Sun Jun 30 13:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90800000000000003,"15min":0.89200000000000002},"min":{"1min":0.61899999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.216,"5min":1.47,"15min":1.47},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.96299999999999997,"15min":0.95499999999999996},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.238,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Sun Jun 30 13:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.89200000000000002,"15min":0.879},"min":{"1min":0.70599999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.87,"15min":0.84999999999999998},"min":{"1min":0.66900000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":1.0309999999999999}]},{"osd":6,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.0840000000000001,"15min":1.0780000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":1.085},{"interface":"front","average":{"1min":1.143,"5min":1.151,"15min":1.1519999999999999},"min":{"1min":0.80400000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.4650000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.1479999999999999}]},{"osd":7,"last update":"Sun Jun 30 13:10:39 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.0980000000000001,"15min":1.0900000000000001},"min":{"1min":0.89600000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":1.131},{"interface":"front","average":{"1min":1.252,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":1.109,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.4610000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.2410000000000001}]}]},{"osd":5,"up_from":36,"seq":154618822694,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5332,"kb_used_data":460,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739820,"statfs":{"total":95995035648,"available":95989575680,"internally_reserved":0,"allocated":471040,"data_stored":188365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.92500000000000004,"15min":0.91900000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.335,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.996,"5min":0.93500000000000005,"15min":0.92500000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.75700000000000001}]},{"osd":1,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.72799999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.093},{"interface":"front","average":{"1min":1.022,"5min":0.96199999999999997,"15min":0.95199999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.90100000000000002}]},{"osd":2,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.96399999999999997,"15min":0.95799999999999996},"min":{"1min":0.77700000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6599999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.149,"5min":1.0349999999999999,"15min":1.016},"min":{"1min":0.86099999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.81799999999999995}]},{"osd":3,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95799999999999996,"15min":0.95299999999999996},"min":{"1min":0.71799999999999997,"5min":0.621,"15min":0.621},"max":{"1min":1.2210000000000001,"5min":1.454,"15min":1.454},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.093,"15min":1.0609999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.7480000000000002,"5min":3.7480000000000002,"15min":3.7480000000000002},"last":0.98999999999999999}]},{"osd":4,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.90900000000000003,"15min":0.90600000000000003},"min":{"1min":0.57799999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5009999999999999,"5min":1.575,"15min":1.575},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.018,"15min":1.018},"min":{"1min":0.77800000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.228,"5min":1.587,"15min":1.587},"last":1.0249999999999999}]},{"osd":6,"last update":"Sun Jun 30 13:10:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1000000000000001,"15min":1.0800000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6000000000000001,"5min":1.659,"15min":1.659},"last":1.159},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.143,"15min":1.1379999999999999},"min":{"1min":0.72299999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5329999999999999,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.2949999999999999}]},{"osd":7,"last update":"Sun Jun 30 13:09:39 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.2050000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-30T13:10:42.565 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph pg dump --format=json 2024-06-30T13:10:43.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:43 smithi144 bash[14758]: audit 2024-06-30T13:10:41.832412+0000 mgr.y (mgr.14144) 272 : audit [DBG] from='client.24494 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-30T13:10:43.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:43 smithi144 bash[14758]: cluster 2024-06-30T13:10:42.410391+0000 mgr.y (mgr.14144) 273 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:43.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:43 smithi144 bash[14758]: audit 2024-06-30T13:10:43.383045+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:43.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:43 smithi144 bash[19242]: audit 2024-06-30T13:10:41.832412+0000 mgr.y (mgr.14144) 272 : audit [DBG] from='client.24494 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-30T13:10:43.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:43 smithi144 bash[19242]: cluster 2024-06-30T13:10:42.410391+0000 mgr.y (mgr.14144) 273 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:43.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:43 smithi144 bash[19242]: audit 2024-06-30T13:10:43.383045+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:43.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:43 smithi167 bash[14619]: audit 2024-06-30T13:10:41.832412+0000 mgr.y (mgr.14144) 272 : audit [DBG] from='client.24494 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-30T13:10:43.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:43 smithi167 bash[14619]: cluster 2024-06-30T13:10:42.410391+0000 mgr.y (mgr.14144) 273 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:43.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:43 smithi167 bash[14619]: audit 2024-06-30T13:10:43.383045+0000 mon.a (mon.0) 508 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:44.355 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:45.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:44 smithi144 bash[19242]: audit 2024-06-30T13:10:43.839211+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:45.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:44 smithi144 bash[19242]: audit 2024-06-30T13:10:44.080395+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:45.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:44 smithi144 bash[19242]: audit 2024-06-30T13:10:44.086559+0000 mon.a (mon.0) 511 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:10:45.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:44 smithi144 bash[19242]: cluster 2024-06-30T13:10:44.411090+0000 mgr.y (mgr.14144) 274 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:45.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:44 smithi144 bash[14758]: audit 2024-06-30T13:10:43.839211+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:45.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:44 smithi144 bash[14758]: audit 2024-06-30T13:10:44.080395+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:45.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:44 smithi144 bash[14758]: audit 2024-06-30T13:10:44.086559+0000 mon.a (mon.0) 511 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:10:45.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:44 smithi144 bash[14758]: cluster 2024-06-30T13:10:44.411090+0000 mgr.y (mgr.14144) 274 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:45.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:44 smithi167 bash[14619]: audit 2024-06-30T13:10:43.839211+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:45.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:44 smithi167 bash[14619]: audit 2024-06-30T13:10:44.080395+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:45.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:44 smithi167 bash[14619]: audit 2024-06-30T13:10:44.086559+0000 mon.a (mon.0) 511 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:10:45.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:44 smithi167 bash[14619]: cluster 2024-06-30T13:10:44.411090+0000 mgr.y (mgr.14144) 274 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:46.009 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:10:45 smithi144 bash[36617]: level=info ts=2024-06-30T13:10:45.933Z caller=cluster.go:640 component=cluster msg="gossip settled; proceeding" elapsed=10.000949243s 2024-06-30T13:10:47.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:47 smithi144 bash[14758]: cluster 2024-06-30T13:10:46.411723+0000 mgr.y (mgr.14144) 275 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:47.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:47 smithi144 bash[19242]: cluster 2024-06-30T13:10:46.411723+0000 mgr.y (mgr.14144) 275 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:47.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:47 smithi167 bash[14619]: cluster 2024-06-30T13:10:46.411723+0000 mgr.y (mgr.14144) 275 : cluster [DBG] pgmap v245: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:48.080 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:10:48.082 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2024-06-30T13:10:48.848 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-06-30T13:10:46.411429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":8,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":749961216,"kb_used":43220,"kb_used_data":3732,"kb_used_omap":0,"kb_used_meta":39424,"kb_avail":749917996,"statfs":{"total":767960285184,"available":767916027904,"internally_reserved":0,"allocated":3821568,"data_stored":1511015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40370176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"50","state":"active+clean","last_fresh":"2024-06-30T13:08:39.496069+0000","last_change":"2024-06-30T13:08:39.496069+0000","last_active":"2024-06-30T13:08:39.496069+0000","last_peered":"2024-06-30T13:08:39.496069+0000","last_clean":"2024-06-30T13:08:39.496069+0000","last_became_active":"2024-06-30T13:08:39.495030+0000","last_became_peered":"2024-06-30T13:08:39.495030+0000","last_unstale":"2024-06-30T13:08:39.496069+0000","last_undegraded":"2024-06-30T13:08:39.496069+0000","last_fullsized":"2024-06-30T13:08:39.496069+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-30T13:06:29.438624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-30T13:06:29.438624+0000","last_clean_scrub_stamp":"2024-06-30T13:06:29.438624+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430237,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5212,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":1.006},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.69499999999999995}]},{"osd":1,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.89900000000000002}]},{"osd":2,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.874},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.095}]},{"osd":3,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.425,"5min":1.425,"15min":1.425},"last":1.206},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.94499999999999995}]},{"osd":4,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":1.288}]},{"osd":5,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":1.262}]},{"osd":6,"last update":"Sun Jun 30 13:09:41 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.365}]}]},{"osd":6,"up_from":42,"seq":180388626466,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5276,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":93739876,"statfs":{"total":95995035648,"available":95989633024,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.91100000000000003,"15min":0.90000000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.93200000000000005,"15min":0.93700000000000006},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.72399999999999998}]},{"osd":1,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.02,"15min":1.0109999999999999},"min":{"1min":0.74099999999999999,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.877},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.92300000000000004,"15min":0.91800000000000004},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.85299999999999998}]},{"osd":2,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95599999999999996,"15min":0.94999999999999996},"min":{"1min":0.69299999999999995,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.464,"5min":1.464,"15min":1.464},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.04,"5min":0.996,"15min":0.98799999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.99099999999999999}]},{"osd":3,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.80700000000000005,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":1.0780000000000001}]},{"osd":4,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.96399999999999997,"15min":0.95999999999999996},"min":{"1min":0.59299999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.052,"5min":1.042,"15min":1.04},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.0549999999999999}]},{"osd":5,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.022,"15min":1.016},"min":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.0980000000000001,"15min":1.089},"min":{"1min":0.80300000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.75700000000000001}]},{"osd":7,"last update":"Sun Jun 30 13:10:44 2024","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.145,"15min":1.1439999999999999},"min":{"1min":0.69999999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.012},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2430000000000001,"15min":1.236},"min":{"1min":1.032,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":1.032}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5532,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":5056,"kb_avail":93739620,"statfs":{"total":95995035648,"available":95989370880,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5177344},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65300000000000002,"15min":0.60599999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.022,"5min":1.024,"15min":1.024},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72599999999999998,"15min":0.64000000000000001},"min":{"1min":0.59499999999999997,"5min":0.441,"15min":0.441},"max":{"1min":1.139,"5min":1.222,"15min":1.222},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Jun 30 13:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.71899999999999997,"15min":0.67100000000000004},"min":{"1min":0.52400000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.83399999999999996,"15min":0.76900000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.94599999999999995}]},{"osd":3,"last update":"Sun Jun 30 13:10:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.85999999999999999,"15min":0.85199999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":1.236},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.93400000000000005}]},{"osd":4,"last update":"Sun Jun 30 13:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.052,"15min":1.0349999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.2549999999999999,"5min":1.554,"15min":1.554},"last":1.046},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.026,"15min":1.026},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.2789999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":1.0760000000000001}]},{"osd":5,"last update":"Sun Jun 30 13:09:49 2024","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.073,"15min":1.077},"min":{"1min":0.77500000000000002,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.997},{"interface":"front","average":{"1min":1.165,"5min":1.1339999999999999,"15min":1.129},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.123}]},{"osd":6,"last update":"Sun Jun 30 13:10:15 2024","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1399999999999999,"15min":1.1519999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.3400000000000001,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.276},{"interface":"front","average":{"1min":1.181,"5min":1.1439999999999999,"15min":1.1379999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.1599999999999999}]},{"osd":7,"last update":"Sun Jun 30 13:10:45 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.216,"15min":1.206},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.224,"5min":1.2130000000000001,"15min":1.2110000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.3600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738434,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5596,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739556,"statfs":{"total":95995035648,"available":95989305344,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jun 30 13:10:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65600000000000003,"15min":0.61299999999999999},"min":{"1min":0.441,"5min":0.435,"15min":0.435},"max":{"1min":0.85499999999999998,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.72099999999999997,"15min":0.65300000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.117,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Jun 30 13:10:37 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.79600000000000004,"15min":0.751},"min":{"1min":0.624,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79900000000000004,"15min":0.75900000000000001},"min":{"1min":0.53400000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.71799999999999997}]},{"osd":3,"last update":"Sun Jun 30 13:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.80400000000000005,"15min":0.79500000000000004},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.2350000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.89700000000000002,"15min":0.88300000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.88200000000000001}]},{"osd":4,"last update":"Sun Jun 30 13:10:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0029999999999999,"15min":0.98899999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":1.127},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.0409999999999999,"15min":1.0309999999999999},"min":{"1min":0.77600000000000002,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.0800000000000001}]},{"osd":5,"last update":"Sun Jun 30 13:09:51 2024","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0309999999999999,"15min":1.0209999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.1379999999999999,"15min":1.1359999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.2490000000000001}]},{"osd":6,"last update":"Sun Jun 30 13:10:17 2024","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.258,"15min":1.2709999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.474,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.385},{"interface":"front","average":{"1min":1.147,"5min":1.234,"15min":1.248},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.425,"5min":1.7130000000000001,"15min":1.7130000000000001},"last":1.0349999999999999}]},{"osd":7,"last update":"Sun Jun 30 13:09:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.331}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5464,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739688,"statfs":{"total":95995035648,"available":95989440512,"internally_reserved":0,"allocated":475136,"data_stored":188680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.71399999999999997,"15min":0.65600000000000003},"min":{"1min":0.52000000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":0.77100000000000002},"min":{"1min":0.52100000000000002,"5min":0.5,"15min":0.5},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.83699999999999997}]},{"osd":1,"last update":"Sun Jun 30 13:10:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69699999999999995,"15min":0.65700000000000003},"min":{"1min":0.46700000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93899999999999995,"5min":1.153,"15min":1.153},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.79500000000000004,"15min":0.73299999999999998},"min":{"1min":0.45800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.1319999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.66300000000000003}]},{"osd":3,"last update":"Sun Jun 30 13:10:01 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.85099999999999998,"15min":0.82299999999999995},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.1699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.82899999999999996,"15min":0.83399999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.464,"15min":1.464},"last":0.68300000000000005}]},{"osd":4,"last update":"Sun Jun 30 13:10:30 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90600000000000003,"15min":0.88700000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.081,"5min":1.667,"15min":1.667},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.073,"15min":1.0369999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4059999999999999,"5min":1.452,"15min":1.452},"last":1.2110000000000001}]},{"osd":5,"last update":"Sun Jun 30 13:09:47 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0640000000000001,"15min":1.0680000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.1060000000000001,"15min":1.0860000000000001},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6379999999999999,"5min":1.6379999999999999,"15min":1.6379999999999999},"last":0.94799999999999995}]},{"osd":6,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.228,"15min":1.2270000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.5549999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1439999999999999,"15min":1.1499999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.4830000000000001,"5min":1.605,"15min":1.605},"last":1.073}]},{"osd":7,"last update":"Sun Jun 30 13:09:42 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":1.018}]}]},{"osd":3,"up_from":25,"seq":107374182448,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5404,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739748,"statfs":{"total":95995035648,"available":95989501952,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.70599999999999996,"15min":0.68400000000000005},"min":{"1min":0.56000000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0269999999999999,"5min":1.179,"15min":1.179},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.80800000000000005,"15min":0.77400000000000002},"min":{"1min":0.62,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.27,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.80800000000000005}]},{"osd":1,"last update":"Sun Jun 30 13:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.746},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.76700000000000002,"15min":0.751},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.997,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.89400000000000002}]},{"osd":2,"last update":"Sun Jun 30 13:10:07 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.80500000000000005,"15min":0.78900000000000003},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1970000000000001,"5min":1.198,"15min":1.198},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.76300000000000001,"15min":0.73899999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.353,"5min":1.353,"15min":1.353},"last":0.53000000000000003}]},{"osd":4,"last update":"Sun Jun 30 13:10:21 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0349999999999999,"15min":1.0229999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.2809999999999999,"5min":1.53,"15min":1.53},"last":0.83599999999999997},{"interface":"front","average":{"1min":1.099,"5min":1.0960000000000001,"15min":1.0840000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.2929999999999999,"5min":1.552,"15min":1.552},"last":0.94599999999999995}]},{"osd":5,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0069999999999999,"15min":1.0009999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.093,"15min":1.087},"min":{"1min":0.88200000000000001,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.877}]},{"osd":6,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.103,"15min":1.103},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.046,"5min":1.0980000000000001,"15min":1.1060000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.0369999999999999}]},{"osd":7,"last update":"Sun Jun 30 13:10:43 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1339999999999999,"15min":1.1399999999999999},"min":{"1min":0.83899999999999997,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.179,"5min":1.2090000000000001,"15min":1.214},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.4259999999999999,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.86199999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5404,"kb_used_data":468,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739748,"statfs":{"total":95995035648,"available":95989501952,"internally_reserved":0,"allocated":479232,"data_stored":188995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.96999999999999997,"15min":0.94399999999999995},"min":{"1min":0.72599999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.999,"5min":1.0289999999999999,"15min":1.006},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.3160000000000001,"5min":1.625,"15min":1.625},"last":0.85899999999999999}]},{"osd":1,"last update":"Sun Jun 30 13:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.88600000000000001,"15min":0.87},"min":{"1min":0.624,"5min":0.503,"15min":0.503},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.93000000000000005,"15min":0.91600000000000004},"min":{"1min":0.67900000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.214,"5min":1.262,"15min":1.262},"last":0.68100000000000005}]},{"osd":2,"last update":"Sun Jun 30 13:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.96799999999999997,"15min":0.96399999999999997},"min":{"1min":0.69899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.95699999999999996,"15min":0.94299999999999995},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.2350000000000001,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Sun Jun 30 13:10:26 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.90800000000000003,"15min":0.89200000000000002},"min":{"1min":0.61899999999999999,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.216,"5min":1.47,"15min":1.47},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.96299999999999997,"15min":0.95499999999999996},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.238,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.76000000000000001}]},{"osd":5,"last update":"Sun Jun 30 13:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.89200000000000002,"15min":0.879},"min":{"1min":0.70599999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.87,"15min":0.84999999999999998},"min":{"1min":0.66900000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.60499999999999998}]},{"osd":6,"last update":"Sun Jun 30 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.0840000000000001,"15min":1.0780000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.151,"15min":1.1519999999999999},"min":{"1min":0.80400000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.4650000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Sun Jun 30 13:10:39 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.0980000000000001,"15min":1.0900000000000001},"min":{"1min":0.89600000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.252,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":1.109,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":1.4610000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":0.871}]}]},{"osd":5,"up_from":36,"seq":154618822695,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5332,"kb_used_data":460,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739820,"statfs":{"total":95995035648,"available":95989575680,"internally_reserved":0,"allocated":471040,"data_stored":188365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.92500000000000004,"15min":0.91900000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.335,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":1.1659999999999999},{"interface":"front","average":{"1min":0.996,"5min":0.93500000000000005,"15min":0.92500000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.014}]},{"osd":1,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.93300000000000005,"15min":0.92400000000000004},"min":{"1min":0.72799999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.022,"5min":0.96199999999999997,"15min":0.95199999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":1.089}]},{"osd":2,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.96399999999999997,"15min":0.95799999999999996},"min":{"1min":0.77700000000000002,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6599999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.131},{"interface":"front","average":{"1min":1.149,"5min":1.0349999999999999,"15min":1.016},"min":{"1min":0.86099999999999999,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":1.0680000000000001}]},{"osd":3,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.95799999999999996,"15min":0.95299999999999996},"min":{"1min":0.71799999999999997,"5min":0.621,"15min":0.621},"max":{"1min":1.2210000000000001,"5min":1.454,"15min":1.454},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.093,"15min":1.0609999999999999},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":3.7480000000000002,"5min":3.7480000000000002,"15min":3.7480000000000002},"last":1.2769999999999999}]},{"osd":4,"last update":"Sun Jun 30 13:09:48 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.90900000000000003,"15min":0.90600000000000003},"min":{"1min":0.57799999999999996,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5009999999999999,"5min":1.575,"15min":1.575},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.018,"15min":1.018},"min":{"1min":0.77800000000000002,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.228,"5min":1.587,"15min":1.587},"last":1.04}]},{"osd":6,"last update":"Sun Jun 30 13:10:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1000000000000001,"15min":1.0800000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6000000000000001,"5min":1.659,"15min":1.659},"last":0.997},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.143,"15min":1.1379999999999999},"min":{"1min":0.72299999999999998,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5329999999999999,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.2130000000000001}]},{"osd":7,"last update":"Sun Jun 30 13:10:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.167,"15min":1.165},"min":{"1min":0.91500000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.542,"5min":1.597,"15min":1.597},"last":1.248},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.347,"15min":1.349},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5429999999999999,"5min":1.726,"15min":1.726},"last":1.3380000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-30T13:10:48.850 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-30T13:10:48.850 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-30T13:10:48.850 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-30T13:10:48.850 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -- ceph health --format=json 2024-06-30T13:10:49.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[19242]: audit 2024-06-30T13:10:47.906923+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:49.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[19242]: audit 2024-06-30T13:10:47.967112+0000 mon.a (mon.0) 513 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:49.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[19242]: audit 2024-06-30T13:10:47.975779+0000 mon.a (mon.0) 514 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:49.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[19242]: audit 2024-06-30T13:10:48.070339+0000 mgr.y (mgr.14144) 276 : audit [DBG] from='client.14577 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-30T13:10:49.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[19242]: cluster 2024-06-30T13:10:48.412356+0000 mgr.y (mgr.14144) 277 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:49.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[14758]: audit 2024-06-30T13:10:47.906923+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:49.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[14758]: audit 2024-06-30T13:10:47.967112+0000 mon.a (mon.0) 513 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:49.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[14758]: audit 2024-06-30T13:10:47.975779+0000 mon.a (mon.0) 514 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:49.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[14758]: audit 2024-06-30T13:10:48.070339+0000 mgr.y (mgr.14144) 276 : audit [DBG] from='client.14577 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-30T13:10:49.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:48 smithi144 bash[14758]: cluster 2024-06-30T13:10:48.412356+0000 mgr.y (mgr.14144) 277 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:49.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:48 smithi167 bash[14619]: audit 2024-06-30T13:10:47.906923+0000 mon.a (mon.0) 512 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:49.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:48 smithi167 bash[14619]: audit 2024-06-30T13:10:47.967112+0000 mon.a (mon.0) 513 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:49.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:48 smithi167 bash[14619]: audit 2024-06-30T13:10:47.975779+0000 mon.a (mon.0) 514 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:10:49.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:48 smithi167 bash[14619]: audit 2024-06-30T13:10:48.070339+0000 mgr.y (mgr.14144) 276 : audit [DBG] from='client.14577 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-30T13:10:49.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:48 smithi167 bash[14619]: cluster 2024-06-30T13:10:48.412356+0000 mgr.y (mgr.14144) 277 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:51.575 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/mon.a/config 2024-06-30T13:10:51.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:51 smithi144 bash[19242]: cluster 2024-06-30T13:10:50.412961+0000 mgr.y (mgr.14144) 278 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:51.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:51 smithi144 bash[14758]: cluster 2024-06-30T13:10:50.412961+0000 mgr.y (mgr.14144) 278 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:51.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:51 smithi167 bash[14619]: cluster 2024-06-30T13:10:50.412961+0000 mgr.y (mgr.14144) 278 : cluster [DBG] pgmap v247: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:53.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:53 smithi144 bash[14758]: cluster 2024-06-30T13:10:52.413567+0000 mgr.y (mgr.14144) 279 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:53.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:53 smithi144 bash[19242]: cluster 2024-06-30T13:10:52.413567+0000 mgr.y (mgr.14144) 279 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:53.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:53 smithi167 bash[14619]: cluster 2024-06-30T13:10:52.413567+0000 mgr.y (mgr.14144) 279 : cluster [DBG] pgmap v248: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:55.018 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-06-30T13:10:55.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:55 smithi144 bash[14758]: cluster 2024-06-30T13:10:54.414253+0000 mgr.y (mgr.14144) 280 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:55.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:55 smithi144 bash[14758]: audit 2024-06-30T13:10:55.016063+0000 mon.c (mon.2) 16 : audit [DBG] from='client.? 172.21.15.144:0/2321505106' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-30T13:10:55.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:55 smithi144 bash[19242]: cluster 2024-06-30T13:10:54.414253+0000 mgr.y (mgr.14144) 280 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:55.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:55 smithi144 bash[19242]: audit 2024-06-30T13:10:55.016063+0000 mon.c (mon.2) 16 : audit [DBG] from='client.? 172.21.15.144:0/2321505106' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-30T13:10:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:55 smithi167 bash[14619]: cluster 2024-06-30T13:10:54.414253+0000 mgr.y (mgr.14144) 280 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:55.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:55 smithi167 bash[14619]: audit 2024-06-30T13:10:55.016063+0000 mon.c (mon.2) 16 : audit [DBG] from='client.? 172.21.15.144:0/2321505106' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-30T13:10:55.778 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-30T13:10:55.779 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-30T13:10:55.779 INFO:tasks.cephadm:Setup complete, yielding 2024-06-30T13:10:55.779 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-30T13:10:55.789 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi144.front.sepia.ceph.com 2024-06-30T13:10:55.790 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'radosgw-admin realm create --rgw-realm=r --default' 2024-06-30T13:10:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:57 smithi144 bash[19242]: cluster 2024-06-30T13:10:56.414864+0000 mgr.y (mgr.14144) 281 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:57 smithi144 bash[19242]: audit 2024-06-30T13:10:56.512965+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:10:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:57 smithi144 bash[19242]: audit 2024-06-30T13:10:56.515447+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:10:57.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:57 smithi144 bash[14758]: cluster 2024-06-30T13:10:56.414864+0000 mgr.y (mgr.14144) 281 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:57.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:57 smithi144 bash[14758]: audit 2024-06-30T13:10:56.512965+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:10:57.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:57 smithi144 bash[14758]: audit 2024-06-30T13:10:56.515447+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:10:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:57 smithi167 bash[14619]: cluster 2024-06-30T13:10:56.414864+0000 mgr.y (mgr.14144) 281 : cluster [DBG] pgmap v250: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:10:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:57 smithi167 bash[14619]: audit 2024-06-30T13:10:56.512965+0000 mon.a (mon.0) 515 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:10:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:57 smithi167 bash[14619]: audit 2024-06-30T13:10:56.515447+0000 mon.a (mon.0) 516 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:11:00.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:10:59 smithi144 bash[14758]: cluster 2024-06-30T13:10:58.415432+0000 mgr.y (mgr.14144) 282 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:11:00.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:10:59 smithi144 bash[19242]: cluster 2024-06-30T13:10:58.415432+0000 mgr.y (mgr.14144) 282 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:11:00.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:10:59 smithi167 bash[14619]: cluster 2024-06-30T13:10:58.415432+0000 mgr.y (mgr.14144) 282 : cluster [DBG] pgmap v251: 1 pgs: 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:11:00.781 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-06-30T13:11:00.781 INFO:teuthology.orchestra.run.smithi144.stdout: "id": "6e2bd9db-1ed3-42ce-8a78-f924f62ad5be", 2024-06-30T13:11:00.781 INFO:teuthology.orchestra.run.smithi144.stdout: "name": "r", 2024-06-30T13:11:00.781 INFO:teuthology.orchestra.run.smithi144.stdout: "current_period": "bc6c5a23-7b4b-4313-9745-c99dadf4f864", 2024-06-30T13:11:00.781 INFO:teuthology.orchestra.run.smithi144.stdout: "epoch": 1 2024-06-30T13:11:00.781 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-06-30T13:11:00.983 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:00 smithi144 bash[14758]: cluster 2024-06-30T13:10:59.730275+0000 mon.a (mon.0) 517 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-06-30T13:11:00.983 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:00 smithi144 bash[14758]: audit 2024-06-30T13:10:59.731488+0000 mon.a (mon.0) 518 : audit [INF] from='client.? 172.21.15.144:0/3644807099' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-30T13:11:00.983 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:00 smithi144 bash[14758]: cluster 2024-06-30T13:11:00.416059+0000 mgr.y (mgr.14144) 283 : cluster [DBG] pgmap v253: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:11:00.984 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:00 smithi144 bash[19242]: cluster 2024-06-30T13:10:59.730275+0000 mon.a (mon.0) 517 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-06-30T13:11:00.984 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:00 smithi144 bash[19242]: audit 2024-06-30T13:10:59.731488+0000 mon.a (mon.0) 518 : audit [INF] from='client.? 172.21.15.144:0/3644807099' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-30T13:11:00.984 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:00 smithi144 bash[19242]: cluster 2024-06-30T13:11:00.416059+0000 mgr.y (mgr.14144) 283 : cluster [DBG] pgmap v253: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:11:01.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:00 smithi167 bash[14619]: cluster 2024-06-30T13:10:59.730275+0000 mon.a (mon.0) 517 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-06-30T13:11:01.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:00 smithi167 bash[14619]: audit 2024-06-30T13:10:59.731488+0000 mon.a (mon.0) 518 : audit [INF] from='client.? 172.21.15.144:0/3644807099' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-06-30T13:11:01.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:00 smithi167 bash[14619]: cluster 2024-06-30T13:11:00.416059+0000 mgr.y (mgr.14144) 283 : cluster [DBG] pgmap v253: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:11:01.572 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'radosgw-admin zonegroup create --rgw-zonegroup=default --master --default' 2024-06-30T13:11:01.862 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:01 smithi144 bash[14758]: audit 2024-06-30T13:11:00.739408+0000 mon.a (mon.0) 519 : audit [INF] from='client.? 172.21.15.144:0/3644807099' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-30T13:11:01.862 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:01 smithi144 bash[14758]: cluster 2024-06-30T13:11:00.739708+0000 mon.a (mon.0) 520 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-06-30T13:11:01.862 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:01 smithi144 bash[19242]: audit 2024-06-30T13:11:00.739408+0000 mon.a (mon.0) 519 : audit [INF] from='client.? 172.21.15.144:0/3644807099' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-30T13:11:01.862 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:01 smithi144 bash[19242]: cluster 2024-06-30T13:11:00.739708+0000 mon.a (mon.0) 520 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-06-30T13:11:02.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:01 smithi167 bash[14619]: audit 2024-06-30T13:11:00.739408+0000 mon.a (mon.0) 519 : audit [INF] from='client.? 172.21.15.144:0/3644807099' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-06-30T13:11:02.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:01 smithi167 bash[14619]: cluster 2024-06-30T13:11:00.739708+0000 mon.a (mon.0) 520 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-06-30T13:11:03.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:02 smithi144 bash[14758]: cluster 2024-06-30T13:11:01.748260+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-06-30T13:11:03.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:02 smithi144 bash[14758]: cluster 2024-06-30T13:11:02.416690+0000 mgr.y (mgr.14144) 284 : cluster [DBG] pgmap v256: 33 pgs: 9 creating+peering, 6 active+clean, 18 unknown; 46 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:11:03.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:02 smithi144 bash[19242]: cluster 2024-06-30T13:11:01.748260+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-06-30T13:11:03.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:02 smithi144 bash[19242]: cluster 2024-06-30T13:11:02.416690+0000 mgr.y (mgr.14144) 284 : cluster [DBG] pgmap v256: 33 pgs: 9 creating+peering, 6 active+clean, 18 unknown; 46 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:11:03.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:02 smithi167 bash[14619]: cluster 2024-06-30T13:11:01.748260+0000 mon.a (mon.0) 521 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-06-30T13:11:03.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:02 smithi167 bash[14619]: cluster 2024-06-30T13:11:02.416690+0000 mgr.y (mgr.14144) 284 : cluster [DBG] pgmap v256: 33 pgs: 9 creating+peering, 6 active+clean, 18 unknown; 46 B data, 42 MiB used, 715 GiB / 715 GiB avail 2024-06-30T13:11:04.904 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-06-30T13:11:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "id": "c1bce6d7-fd74-4794-87ca-6978558529b8", 2024-06-30T13:11:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "name": "default", 2024-06-30T13:11:04.904 INFO:teuthology.orchestra.run.smithi144.stdout: "api_name": "default", 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "is_master": "true", 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "endpoints": [], 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "hostnames": [], 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "hostnames_s3website": [], 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "master_zone": "", 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "zones": [], 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "placement_targets": [], 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "default_placement": "", 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "realm_id": "6e2bd9db-1ed3-42ce-8a78-f924f62ad5be", 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "sync_policy": { 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: "groups": [] 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:04.905 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-06-30T13:11:05.621 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=z --master --default' 2024-06-30T13:11:05.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:05 smithi144 bash[14758]: cluster 2024-06-30T13:11:04.417425+0000 mgr.y (mgr.14144) 285 : cluster [DBG] pgmap v257: 33 pgs: 9 creating+peering, 18 active+clean, 6 unknown; 470 B data, 43 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s wr, 2 op/s 2024-06-30T13:11:05.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:05 smithi144 bash[19242]: cluster 2024-06-30T13:11:04.417425+0000 mgr.y (mgr.14144) 285 : cluster [DBG] pgmap v257: 33 pgs: 9 creating+peering, 18 active+clean, 6 unknown; 470 B data, 43 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s wr, 2 op/s 2024-06-30T13:11:05.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:05 smithi167 bash[14619]: cluster 2024-06-30T13:11:04.417425+0000 mgr.y (mgr.14144) 285 : cluster [DBG] pgmap v257: 33 pgs: 9 creating+peering, 18 active+clean, 6 unknown; 470 B data, 43 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s wr, 2 op/s 2024-06-30T13:11:06.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:06 smithi144 bash[14758]: cluster 2024-06-30T13:11:06.418173+0000 mgr.y (mgr.14144) 286 : cluster [DBG] pgmap v258: 33 pgs: 33 active+clean; 516 B data, 43 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-30T13:11:06.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:06 smithi144 bash[19242]: cluster 2024-06-30T13:11:06.418173+0000 mgr.y (mgr.14144) 286 : cluster [DBG] pgmap v258: 33 pgs: 33 active+clean; 516 B data, 43 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-30T13:11:06.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:06 smithi167 bash[14619]: cluster 2024-06-30T13:11:06.418173+0000 mgr.y (mgr.14144) 286 : cluster [DBG] pgmap v258: 33 pgs: 33 active+clean; 516 B data, 43 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-06-30T13:11:09.121 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-06-30T13:11:09.121 INFO:teuthology.orchestra.run.smithi144.stdout: "id": "a96b0e60-740a-48b1-adec-782abb32fec2", 2024-06-30T13:11:09.121 INFO:teuthology.orchestra.run.smithi144.stdout: "name": "z", 2024-06-30T13:11:09.121 INFO:teuthology.orchestra.run.smithi144.stdout: "domain_root": "z.rgw.meta:root", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "control_pool": "z.rgw.control", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "gc_pool": "z.rgw.log:gc", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "lc_pool": "z.rgw.log:lc", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "log_pool": "z.rgw.log", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "intent_log_pool": "z.rgw.log:intent", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "usage_log_pool": "z.rgw.log:usage", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "roles_pool": "z.rgw.meta:roles", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "reshard_pool": "z.rgw.log:reshard", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "user_keys_pool": "z.rgw.meta:users.keys", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "user_email_pool": "z.rgw.meta:users.email", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "user_swift_pool": "z.rgw.meta:users.swift", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "user_uid_pool": "z.rgw.meta:users.uid", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "otp_pool": "z.rgw.otp", 2024-06-30T13:11:09.122 INFO:teuthology.orchestra.run.smithi144.stdout: "system_key": { 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: "access_key": "", 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: "secret_key": "" 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: "placement_pools": [ 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "default-placement", 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: "val": { 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: "index_pool": "z.rgw.buckets.index", 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: "storage_classes": { 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: "STANDARD": { 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: "data_pool": "z.rgw.buckets.data" 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:09.123 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:11:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: "data_extra_pool": "z.rgw.buckets.non-ec", 2024-06-30T13:11:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: "index_type": 0 2024-06-30T13:11:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-06-30T13:11:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: "realm_id": "6e2bd9db-1ed3-42ce-8a78-f924f62ad5be", 2024-06-30T13:11:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: "notif_pool": "z.rgw.log:notif" 2024-06-30T13:11:09.124 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-06-30T13:11:09.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:09 smithi144 bash[19242]: cluster 2024-06-30T13:11:08.418899+0000 mgr.y (mgr.14144) 287 : cluster [DBG] pgmap v259: 33 pgs: 33 active+clean; 680 B data, 43 MiB used, 715 GiB / 715 GiB avail; 153 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-06-30T13:11:09.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:09 smithi144 bash[14758]: cluster 2024-06-30T13:11:08.418899+0000 mgr.y (mgr.14144) 287 : cluster [DBG] pgmap v259: 33 pgs: 33 active+clean; 680 B data, 43 MiB used, 715 GiB / 715 GiB avail; 153 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-06-30T13:11:09.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:09 smithi167 bash[14619]: cluster 2024-06-30T13:11:08.418899+0000 mgr.y (mgr.14144) 287 : cluster [DBG] pgmap v259: 33 pgs: 33 active+clean; 680 B data, 43 MiB used, 715 GiB / 715 GiB avail; 153 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-06-30T13:11:09.862 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'radosgw-admin period update --rgw-realm=r --commit' 2024-06-30T13:11:11.009 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:10 smithi144 bash[14758]: cluster 2024-06-30T13:11:10.419691+0000 mgr.y (mgr.14144) 288 : cluster [DBG] pgmap v260: 33 pgs: 33 active+clean; 772 B data, 43 MiB used, 715 GiB / 715 GiB avail; 354 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-06-30T13:11:11.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:10 smithi144 bash[19242]: cluster 2024-06-30T13:11:10.419691+0000 mgr.y (mgr.14144) 288 : cluster [DBG] pgmap v260: 33 pgs: 33 active+clean; 772 B data, 43 MiB used, 715 GiB / 715 GiB avail; 354 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-06-30T13:11:11.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:10 smithi167 bash[14619]: cluster 2024-06-30T13:11:10.419691+0000 mgr.y (mgr.14144) 288 : cluster [DBG] pgmap v260: 33 pgs: 33 active+clean; 772 B data, 43 MiB used, 715 GiB / 715 GiB avail; 354 B/s rd, 1.0 KiB/s wr, 1 op/s 2024-06-30T13:11:14.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:13 smithi167 bash[14619]: cluster 2024-06-30T13:11:12.420423+0000 mgr.y (mgr.14144) 289 : cluster [DBG] pgmap v261: 33 pgs: 33 active+clean; 1.1 KiB data, 43 MiB used, 715 GiB / 715 GiB avail; 383 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-06-30T13:11:14.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:13 smithi144 bash[19242]: cluster 2024-06-30T13:11:12.420423+0000 mgr.y (mgr.14144) 289 : cluster [DBG] pgmap v261: 33 pgs: 33 active+clean; 1.1 KiB data, 43 MiB used, 715 GiB / 715 GiB avail; 383 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-06-30T13:11:14.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:13 smithi144 bash[14758]: cluster 2024-06-30T13:11:12.420423+0000 mgr.y (mgr.14144) 289 : cluster [DBG] pgmap v261: 33 pgs: 33 active+clean; 1.1 KiB data, 43 MiB used, 715 GiB / 715 GiB avail; 383 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-06-30T13:11:15.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:14 smithi144 bash[19242]: cluster 2024-06-30T13:11:13.814570+0000 mon.a (mon.0) 522 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-06-30T13:11:15.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:14 smithi144 bash[19242]: audit 2024-06-30T13:11:13.818073+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.144:0/2832321833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-06-30T13:11:15.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:14 smithi144 bash[19242]: audit 2024-06-30T13:11:13.818773+0000 mon.a (mon.0) 523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-06-30T13:11:15.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:14 smithi144 bash[19242]: cluster 2024-06-30T13:11:14.421135+0000 mgr.y (mgr.14144) 290 : cluster [DBG] pgmap v263: 65 pgs: 32 unknown, 33 active+clean; 1.8 KiB data, 43 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 921 B/s wr, 2 op/s 2024-06-30T13:11:15.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:14 smithi144 bash[14758]: cluster 2024-06-30T13:11:13.814570+0000 mon.a (mon.0) 522 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-06-30T13:11:15.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:14 smithi144 bash[14758]: audit 2024-06-30T13:11:13.818073+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.144:0/2832321833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-06-30T13:11:15.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:14 smithi144 bash[14758]: audit 2024-06-30T13:11:13.818773+0000 mon.a (mon.0) 523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-06-30T13:11:15.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:14 smithi144 bash[14758]: cluster 2024-06-30T13:11:14.421135+0000 mgr.y (mgr.14144) 290 : cluster [DBG] pgmap v263: 65 pgs: 32 unknown, 33 active+clean; 1.8 KiB data, 43 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 921 B/s wr, 2 op/s 2024-06-30T13:11:15.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:14 smithi167 bash[14619]: cluster 2024-06-30T13:11:13.814570+0000 mon.a (mon.0) 522 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-06-30T13:11:15.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:14 smithi167 bash[14619]: audit 2024-06-30T13:11:13.818073+0000 mon.c (mon.2) 17 : audit [INF] from='client.? 172.21.15.144:0/2832321833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-06-30T13:11:15.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:14 smithi167 bash[14619]: audit 2024-06-30T13:11:13.818773+0000 mon.a (mon.0) 523 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]: dispatch 2024-06-30T13:11:15.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:14 smithi167 bash[14619]: cluster 2024-06-30T13:11:14.421135+0000 mgr.y (mgr.14144) 290 : cluster [DBG] pgmap v263: 65 pgs: 32 unknown, 33 active+clean; 1.8 KiB data, 43 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 921 B/s wr, 2 op/s 2024-06-30T13:11:16.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:15 smithi144 bash[14758]: audit 2024-06-30T13:11:14.819417+0000 mon.a (mon.0) 524 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-06-30T13:11:16.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:15 smithi144 bash[14758]: cluster 2024-06-30T13:11:14.819778+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-06-30T13:11:16.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:15 smithi144 bash[19242]: audit 2024-06-30T13:11:14.819417+0000 mon.a (mon.0) 524 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-06-30T13:11:16.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:15 smithi144 bash[19242]: cluster 2024-06-30T13:11:14.819778+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-06-30T13:11:16.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:15 smithi167 bash[14619]: audit 2024-06-30T13:11:14.819417+0000 mon.a (mon.0) 524 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.log","app": "rgw"}]': finished 2024-06-30T13:11:16.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:15 smithi167 bash[14619]: cluster 2024-06-30T13:11:14.819778+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-06-30T13:11:17.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:16 smithi144 bash[19242]: cluster 2024-06-30T13:11:15.848842+0000 mon.a (mon.0) 526 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-06-30T13:11:17.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:16 smithi144 bash[19242]: audit 2024-06-30T13:11:15.850894+0000 mon.c (mon.2) 18 : audit [INF] from='client.? 172.21.15.144:0/2832321833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-06-30T13:11:17.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:16 smithi144 bash[19242]: audit 2024-06-30T13:11:15.852322+0000 mon.a (mon.0) 527 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-06-30T13:11:17.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:16 smithi144 bash[19242]: cluster 2024-06-30T13:11:16.421909+0000 mgr.y (mgr.14144) 291 : cluster [DBG] pgmap v266: 97 pgs: 6 creating+peering, 49 unknown, 42 active+clean; 1.8 KiB data, 43 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 3 op/s 2024-06-30T13:11:17.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:16 smithi144 bash[14758]: cluster 2024-06-30T13:11:15.848842+0000 mon.a (mon.0) 526 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-06-30T13:11:17.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:16 smithi144 bash[14758]: audit 2024-06-30T13:11:15.850894+0000 mon.c (mon.2) 18 : audit [INF] from='client.? 172.21.15.144:0/2832321833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-06-30T13:11:17.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:16 smithi144 bash[14758]: audit 2024-06-30T13:11:15.852322+0000 mon.a (mon.0) 527 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-06-30T13:11:17.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:16 smithi144 bash[14758]: cluster 2024-06-30T13:11:16.421909+0000 mgr.y (mgr.14144) 291 : cluster [DBG] pgmap v266: 97 pgs: 6 creating+peering, 49 unknown, 42 active+clean; 1.8 KiB data, 43 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 3 op/s 2024-06-30T13:11:17.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:16 smithi167 bash[14619]: cluster 2024-06-30T13:11:15.848842+0000 mon.a (mon.0) 526 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-06-30T13:11:17.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:16 smithi167 bash[14619]: audit 2024-06-30T13:11:15.850894+0000 mon.c (mon.2) 18 : audit [INF] from='client.? 172.21.15.144:0/2832321833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-06-30T13:11:17.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:16 smithi167 bash[14619]: audit 2024-06-30T13:11:15.852322+0000 mon.a (mon.0) 527 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]: dispatch 2024-06-30T13:11:17.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:16 smithi167 bash[14619]: cluster 2024-06-30T13:11:16.421909+0000 mgr.y (mgr.14144) 291 : cluster [DBG] pgmap v266: 97 pgs: 6 creating+peering, 49 unknown, 42 active+clean; 1.8 KiB data, 43 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 3 op/s 2024-06-30T13:11:18.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:17 smithi144 bash[19242]: audit 2024-06-30T13:11:16.858276+0000 mon.a (mon.0) 528 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-06-30T13:11:18.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:17 smithi144 bash[19242]: cluster 2024-06-30T13:11:16.858618+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-06-30T13:11:18.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:17 smithi144 bash[14758]: audit 2024-06-30T13:11:16.858276+0000 mon.a (mon.0) 528 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-06-30T13:11:18.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:17 smithi144 bash[14758]: cluster 2024-06-30T13:11:16.858618+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-06-30T13:11:18.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:17 smithi167 bash[14619]: audit 2024-06-30T13:11:16.858276+0000 mon.a (mon.0) 528 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.control","app": "rgw"}]': finished 2024-06-30T13:11:18.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:17 smithi167 bash[14619]: cluster 2024-06-30T13:11:16.858618+0000 mon.a (mon.0) 529 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-06-30T13:11:19.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:18 smithi144 bash[14758]: cluster 2024-06-30T13:11:17.861568+0000 mon.a (mon.0) 530 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-06-30T13:11:19.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:18 smithi144 bash[14758]: cluster 2024-06-30T13:11:18.422592+0000 mgr.y (mgr.14144) 292 : cluster [DBG] pgmap v269: 97 pgs: 8 creating+peering, 31 unknown, 58 active+clean; 2.1 KiB data, 44 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-30T13:11:19.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:18 smithi144 bash[19242]: cluster 2024-06-30T13:11:17.861568+0000 mon.a (mon.0) 530 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-06-30T13:11:19.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:18 smithi144 bash[19242]: cluster 2024-06-30T13:11:18.422592+0000 mgr.y (mgr.14144) 292 : cluster [DBG] pgmap v269: 97 pgs: 8 creating+peering, 31 unknown, 58 active+clean; 2.1 KiB data, 44 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-30T13:11:19.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:18 smithi167 bash[14619]: cluster 2024-06-30T13:11:17.861568+0000 mon.a (mon.0) 530 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-06-30T13:11:19.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:18 smithi167 bash[14619]: cluster 2024-06-30T13:11:18.422592+0000 mgr.y (mgr.14144) 292 : cluster [DBG] pgmap v269: 97 pgs: 8 creating+peering, 31 unknown, 58 active+clean; 2.1 KiB data, 44 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-06-30T13:11:20.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:19 smithi144 bash[14758]: cluster 2024-06-30T13:11:18.880530+0000 mon.a (mon.0) 531 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-06-30T13:11:20.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:19 smithi144 bash[14758]: audit 2024-06-30T13:11:18.884789+0000 mon.c (mon.2) 19 : audit [INF] from='client.? 172.21.15.144:0/1736423966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-30T13:11:20.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:19 smithi144 bash[14758]: audit 2024-06-30T13:11:18.888243+0000 mon.a (mon.0) 532 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-30T13:11:20.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:19 smithi144 bash[19242]: cluster 2024-06-30T13:11:18.880530+0000 mon.a (mon.0) 531 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-06-30T13:11:20.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:19 smithi144 bash[19242]: audit 2024-06-30T13:11:18.884789+0000 mon.c (mon.2) 19 : audit [INF] from='client.? 172.21.15.144:0/1736423966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-30T13:11:20.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:19 smithi144 bash[19242]: audit 2024-06-30T13:11:18.888243+0000 mon.a (mon.0) 532 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-30T13:11:20.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:19 smithi167 bash[14619]: cluster 2024-06-30T13:11:18.880530+0000 mon.a (mon.0) 531 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-06-30T13:11:20.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:19 smithi167 bash[14619]: audit 2024-06-30T13:11:18.884789+0000 mon.c (mon.2) 19 : audit [INF] from='client.? 172.21.15.144:0/1736423966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-30T13:11:20.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:19 smithi167 bash[14619]: audit 2024-06-30T13:11:18.888243+0000 mon.a (mon.0) 532 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]: dispatch 2024-06-30T13:11:21.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[14758]: audit 2024-06-30T13:11:19.885987+0000 mon.a (mon.0) 533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-06-30T13:11:21.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[14758]: cluster 2024-06-30T13:11:19.886298+0000 mon.a (mon.0) 534 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-06-30T13:11:21.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[14758]: audit 2024-06-30T13:11:19.902649+0000 mon.c (mon.2) 20 : audit [INF] from='client.? 172.21.15.144:0/1736423966' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-30T13:11:21.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[14758]: audit 2024-06-30T13:11:19.903145+0000 mon.a (mon.0) 535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-30T13:11:21.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[14758]: cluster 2024-06-30T13:11:20.423334+0000 mgr.y (mgr.14144) 293 : cluster [DBG] pgmap v272: 129 pgs: 9 creating+peering, 37 unknown, 83 active+clean; 9.2 KiB data, 45 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 12 KiB/s wr, 48 op/s 2024-06-30T13:11:21.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[14758]: audit 2024-06-30T13:11:20.889148+0000 mon.a (mon.0) 536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-30T13:11:21.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[14758]: cluster 2024-06-30T13:11:20.889324+0000 mon.a (mon.0) 537 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[14758]: audit 2024-06-30T13:11:20.891070+0000 mon.c (mon.2) 21 : audit [INF] from='client.? 172.21.15.144:0/1736423966' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[14758]: audit 2024-06-30T13:11:20.891627+0000 mon.a (mon.0) 538 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[19242]: audit 2024-06-30T13:11:19.885987+0000 mon.a (mon.0) 533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[19242]: cluster 2024-06-30T13:11:19.886298+0000 mon.a (mon.0) 534 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[19242]: audit 2024-06-30T13:11:19.902649+0000 mon.c (mon.2) 20 : audit [INF] from='client.? 172.21.15.144:0/1736423966' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[19242]: audit 2024-06-30T13:11:19.903145+0000 mon.a (mon.0) 535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[19242]: cluster 2024-06-30T13:11:20.423334+0000 mgr.y (mgr.14144) 293 : cluster [DBG] pgmap v272: 129 pgs: 9 creating+peering, 37 unknown, 83 active+clean; 9.2 KiB data, 45 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 12 KiB/s wr, 48 op/s 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[19242]: audit 2024-06-30T13:11:20.889148+0000 mon.a (mon.0) 536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[19242]: cluster 2024-06-30T13:11:20.889324+0000 mon.a (mon.0) 537 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[19242]: audit 2024-06-30T13:11:20.891070+0000 mon.c (mon.2) 21 : audit [INF] from='client.? 172.21.15.144:0/1736423966' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-30T13:11:21.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:20 smithi144 bash[19242]: audit 2024-06-30T13:11:20.891627+0000 mon.a (mon.0) 538 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-30T13:11:21.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:20 smithi167 bash[14619]: audit 2024-06-30T13:11:19.885987+0000 mon.a (mon.0) 533 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "z.rgw.meta","app": "rgw"}]': finished 2024-06-30T13:11:21.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:20 smithi167 bash[14619]: cluster 2024-06-30T13:11:19.886298+0000 mon.a (mon.0) 534 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-06-30T13:11:21.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:20 smithi167 bash[14619]: audit 2024-06-30T13:11:19.902649+0000 mon.c (mon.2) 20 : audit [INF] from='client.? 172.21.15.144:0/1736423966' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-30T13:11:21.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:20 smithi167 bash[14619]: audit 2024-06-30T13:11:19.903145+0000 mon.a (mon.0) 535 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-06-30T13:11:21.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:20 smithi167 bash[14619]: cluster 2024-06-30T13:11:20.423334+0000 mgr.y (mgr.14144) 293 : cluster [DBG] pgmap v272: 129 pgs: 9 creating+peering, 37 unknown, 83 active+clean; 9.2 KiB data, 45 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 12 KiB/s wr, 48 op/s 2024-06-30T13:11:21.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:20 smithi167 bash[14619]: audit 2024-06-30T13:11:20.889148+0000 mon.a (mon.0) 536 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-06-30T13:11:21.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:20 smithi167 bash[14619]: cluster 2024-06-30T13:11:20.889324+0000 mon.a (mon.0) 537 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-06-30T13:11:21.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:20 smithi167 bash[14619]: audit 2024-06-30T13:11:20.891070+0000 mon.c (mon.2) 21 : audit [INF] from='client.? 172.21.15.144:0/1736423966' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-30T13:11:21.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:20 smithi167 bash[14619]: audit 2024-06-30T13:11:20.891627+0000 mon.a (mon.0) 538 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]: dispatch 2024-06-30T13:11:22.086 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-06-30T13:11:22.086 INFO:teuthology.orchestra.run.smithi144.stdout: "id": "92d39c54-d799-406a-a1e2-aab8a8e1a639", 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "epoch": 1, 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "predecessor_uuid": "bc6c5a23-7b4b-4313-9745-c99dadf4f864", 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "sync_status": [], 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "period_map": { 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "id": "92d39c54-d799-406a-a1e2-aab8a8e1a639", 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "zonegroups": [ 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "id": "c1bce6d7-fd74-4794-87ca-6978558529b8", 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "name": "default", 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "api_name": "default", 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "is_master": "true", 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "endpoints": [], 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "hostnames": [], 2024-06-30T13:11:22.087 INFO:teuthology.orchestra.run.smithi144.stdout: "hostnames_s3website": [], 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "master_zone": "a96b0e60-740a-48b1-adec-782abb32fec2", 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "zones": [ 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "id": "a96b0e60-740a-48b1-adec-782abb32fec2", 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "name": "z", 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "endpoints": [], 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "log_meta": "false", 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "log_data": "false", 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "bucket_index_max_shards": 11, 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "read_only": "false", 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "tier_type": "", 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "sync_from_all": "true", 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "sync_from": [], 2024-06-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi144.stdout: "redirect_zone": "" 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "placement_targets": [ 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "name": "default-placement", 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "tags": [], 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "storage_classes": [ 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "STANDARD" 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "default_placement": "default-placement", 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "realm_id": "6e2bd9db-1ed3-42ce-8a78-f924f62ad5be", 2024-06-30T13:11:22.089 INFO:teuthology.orchestra.run.smithi144.stdout: "sync_policy": { 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "groups": [] 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "short_zone_ids": [ 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: { 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "key": "a96b0e60-740a-48b1-adec-782abb32fec2", 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "val": 3908613896 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "master_zonegroup": "c1bce6d7-fd74-4794-87ca-6978558529b8", 2024-06-30T13:11:22.090 INFO:teuthology.orchestra.run.smithi144.stdout: "master_zone": "a96b0e60-740a-48b1-adec-782abb32fec2", 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "period_config": { 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "bucket_quota": { 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "enabled": false, 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "check_on_raw": false, 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "max_size": -1, 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "max_size_kb": 0, 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "max_objects": -1 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "user_quota": { 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "enabled": false, 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "check_on_raw": false, 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "max_size": -1, 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "max_size_kb": 0, 2024-06-30T13:11:22.091 INFO:teuthology.orchestra.run.smithi144.stdout: "max_objects": -1 2024-06-30T13:11:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:11:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:11:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: "realm_id": "6e2bd9db-1ed3-42ce-8a78-f924f62ad5be", 2024-06-30T13:11:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: "realm_name": "r", 2024-06-30T13:11:22.092 INFO:teuthology.orchestra.run.smithi144.stdout: "realm_epoch": 2 2024-06-30T13:11:22.092 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-06-30T13:11:23.106 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph orch apply rgw foo --realm r --zone z --placement=2 --port=8000' 2024-06-30T13:11:23.259 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:22 smithi144 bash[14758]: audit 2024-06-30T13:11:21.891410+0000 mon.a (mon.0) 539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2024-06-30T13:11:23.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:22 smithi144 bash[14758]: cluster 2024-06-30T13:11:21.891650+0000 mon.a (mon.0) 540 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-06-30T13:11:23.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:22 smithi144 bash[14758]: cluster 2024-06-30T13:11:22.424080+0000 mgr.y (mgr.14144) 294 : cluster [DBG] pgmap v275: 129 pgs: 7 creating+peering, 12 unknown, 110 active+clean; 18 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 26 KiB/s wr, 103 op/s 2024-06-30T13:11:23.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:22 smithi144 bash[19242]: audit 2024-06-30T13:11:21.891410+0000 mon.a (mon.0) 539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2024-06-30T13:11:23.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:22 smithi144 bash[19242]: cluster 2024-06-30T13:11:21.891650+0000 mon.a (mon.0) 540 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-06-30T13:11:23.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:22 smithi144 bash[19242]: cluster 2024-06-30T13:11:22.424080+0000 mgr.y (mgr.14144) 294 : cluster [DBG] pgmap v275: 129 pgs: 7 creating+peering, 12 unknown, 110 active+clean; 18 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 26 KiB/s wr, 103 op/s 2024-06-30T13:11:23.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:22 smithi167 bash[14619]: audit 2024-06-30T13:11:21.891410+0000 mon.a (mon.0) 539 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "z.rgw.meta", "var": "pg_num_min", "val": "8"}]': finished 2024-06-30T13:11:23.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:22 smithi167 bash[14619]: cluster 2024-06-30T13:11:21.891650+0000 mon.a (mon.0) 540 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-06-30T13:11:23.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:22 smithi167 bash[14619]: cluster 2024-06-30T13:11:22.424080+0000 mgr.y (mgr.14144) 294 : cluster [DBG] pgmap v275: 129 pgs: 7 creating+peering, 12 unknown, 110 active+clean; 18 KiB data, 47 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 26 KiB/s wr, 103 op/s 2024-06-30T13:11:24.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:24 smithi144 bash[14758]: cluster 2024-06-30T13:11:24.424916+0000 mgr.y (mgr.14144) 295 : cluster [DBG] pgmap v276: 129 pgs: 7 creating+peering, 122 active+clean; 23 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 21 KiB/s wr, 83 op/s 2024-06-30T13:11:24.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:24 smithi144 bash[19242]: cluster 2024-06-30T13:11:24.424916+0000 mgr.y (mgr.14144) 295 : cluster [DBG] pgmap v276: 129 pgs: 7 creating+peering, 122 active+clean; 23 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 21 KiB/s wr, 83 op/s 2024-06-30T13:11:25.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:24 smithi167 bash[14619]: cluster 2024-06-30T13:11:24.424916+0000 mgr.y (mgr.14144) 295 : cluster [DBG] pgmap v276: 129 pgs: 7 creating+peering, 122 active+clean; 23 KiB data, 48 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 21 KiB/s wr, 83 op/s 2024-06-30T13:11:26.759 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:26 smithi144 bash[14758]: cluster 2024-06-30T13:11:26.425856+0000 mgr.y (mgr.14144) 296 : cluster [DBG] pgmap v277: 129 pgs: 129 active+clean; 26 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 17 KiB/s wr, 68 op/s 2024-06-30T13:11:26.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:26 smithi144 bash[19242]: cluster 2024-06-30T13:11:26.425856+0000 mgr.y (mgr.14144) 296 : cluster [DBG] pgmap v277: 129 pgs: 129 active+clean; 26 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 17 KiB/s wr, 68 op/s 2024-06-30T13:11:26.798 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled rgw.foo update... 2024-06-30T13:11:27.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:26 smithi167 bash[14619]: cluster 2024-06-30T13:11:26.425856+0000 mgr.y (mgr.14144) 296 : cluster [DBG] pgmap v277: 129 pgs: 129 active+clean; 26 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 13 KiB/s rd, 17 KiB/s wr, 68 op/s 2024-06-30T13:11:27.695 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph orch apply rgw smpl' 2024-06-30T13:11:28.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:26.787714+0000 mgr.y (mgr.14144) 297 : audit [DBG] from='client.14649 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:11:28.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: cephadm 2024-06-30T13:11:26.790146+0000 mgr.y (mgr.14144) 298 : cephadm [INF] Saving service rgw.foo spec with placement count:2 2024-06-30T13:11:28.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:26.794825+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:26.795969+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:27.051016+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:27.065294+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:27.074292+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: cephadm 2024-06-30T13:11:27.075326+0000 mgr.y (mgr.14144) 299 : cephadm [INF] Saving service rgw.foo spec with placement count:2 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:27.082341+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:27.083576+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi144.pwxrka", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:27.090561+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi144.pwxrka", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:27.101448+0000 mon.a (mon.0) 549 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: audit 2024-06-30T13:11:27.102621+0000 mon.a (mon.0) 550 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:28.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[14758]: cephadm 2024-06-30T13:11:27.103916+0000 mgr.y (mgr.14144) 300 : cephadm [INF] Deploying daemon rgw.foo.smithi144.pwxrka on smithi144 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:26.787714+0000 mgr.y (mgr.14144) 297 : audit [DBG] from='client.14649 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: cephadm 2024-06-30T13:11:26.790146+0000 mgr.y (mgr.14144) 298 : cephadm [INF] Saving service rgw.foo spec with placement count:2 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:26.794825+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:26.795969+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:27.051016+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:27.065294+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:27.074292+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: cephadm 2024-06-30T13:11:27.075326+0000 mgr.y (mgr.14144) 299 : cephadm [INF] Saving service rgw.foo spec with placement count:2 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:27.082341+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:27.083576+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi144.pwxrka", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:27.090561+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi144.pwxrka", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:27.101448+0000 mon.a (mon.0) 549 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: audit 2024-06-30T13:11:27.102621+0000 mon.a (mon.0) 550 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:28.013 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:27 smithi144 bash[19242]: cephadm 2024-06-30T13:11:27.103916+0000 mgr.y (mgr.14144) 300 : cephadm [INF] Deploying daemon rgw.foo.smithi144.pwxrka on smithi144 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:26.787714+0000 mgr.y (mgr.14144) 297 : audit [DBG] from='client.14649 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo", "realm": "r", "zone": "z", "placement": "2", "port": 8000, "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: cephadm 2024-06-30T13:11:26.790146+0000 mgr.y (mgr.14144) 298 : cephadm [INF] Saving service rgw.foo spec with placement count:2 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:26.794825+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:26.795969+0000 mon.a (mon.0) 542 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:27.051016+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:27.065294+0000 mon.a (mon.0) 544 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:27.074292+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: cephadm 2024-06-30T13:11:27.075326+0000 mgr.y (mgr.14144) 299 : cephadm [INF] Saving service rgw.foo spec with placement count:2 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:27.082341+0000 mon.a (mon.0) 546 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:27.083576+0000 mon.a (mon.0) 547 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi144.pwxrka", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:27.090561+0000 mon.a (mon.0) 548 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi144.pwxrka", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:28.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:27.101448+0000 mon.a (mon.0) 549 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:28.279 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: audit 2024-06-30T13:11:27.102621+0000 mon.a (mon.0) 550 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:28.279 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:27 smithi167 bash[14619]: cephadm 2024-06-30T13:11:27.103916+0000 mgr.y (mgr.14144) 300 : cephadm [INF] Deploying daemon rgw.foo.smithi144.pwxrka on smithi144 2024-06-30T13:11:29.259 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:28 smithi144 bash[19242]: cluster 2024-06-30T13:11:28.426860+0000 mgr.y (mgr.14144) 301 : cluster [DBG] pgmap v278: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 15 KiB/s wr, 58 op/s 2024-06-30T13:11:29.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:28 smithi144 bash[14758]: cluster 2024-06-30T13:11:28.426860+0000 mgr.y (mgr.14144) 301 : cluster [DBG] pgmap v278: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 15 KiB/s wr, 58 op/s 2024-06-30T13:11:29.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:28 smithi167 bash[14619]: cluster 2024-06-30T13:11:28.426860+0000 mgr.y (mgr.14144) 301 : cluster [DBG] pgmap v278: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 11 KiB/s rd, 15 KiB/s wr, 58 op/s 2024-06-30T13:11:31.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:30 smithi144 bash[14758]: cluster 2024-06-30T13:11:30.427942+0000 mgr.y (mgr.14144) 302 : cluster [DBG] pgmap v279: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 8.2 KiB/s wr, 31 op/s 2024-06-30T13:11:31.012 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:30 smithi144 bash[19242]: cluster 2024-06-30T13:11:30.427942+0000 mgr.y (mgr.14144) 302 : cluster [DBG] pgmap v279: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 8.2 KiB/s wr, 31 op/s 2024-06-30T13:11:31.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:30 smithi167 bash[14619]: cluster 2024-06-30T13:11:30.427942+0000 mgr.y (mgr.14144) 302 : cluster [DBG] pgmap v279: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 4.8 KiB/s rd, 8.2 KiB/s wr, 31 op/s 2024-06-30T13:11:32.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[19242]: audit 2024-06-30T13:11:31.432566+0000 mon.a (mon.0) 551 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:32.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[19242]: audit 2024-06-30T13:11:31.433807+0000 mon.a (mon.0) 552 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi167.ojckkr", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:32.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[19242]: audit 2024-06-30T13:11:31.438442+0000 mon.a (mon.0) 553 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi167.ojckkr", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:32.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[19242]: audit 2024-06-30T13:11:31.444888+0000 mon.a (mon.0) 554 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:32.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[19242]: audit 2024-06-30T13:11:31.445956+0000 mon.a (mon.0) 555 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:32.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[19242]: cephadm 2024-06-30T13:11:31.446940+0000 mgr.y (mgr.14144) 303 : cephadm [INF] Deploying daemon rgw.foo.smithi167.ojckkr on smithi167 2024-06-30T13:11:32.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[14758]: audit 2024-06-30T13:11:31.432566+0000 mon.a (mon.0) 551 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:32.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[14758]: audit 2024-06-30T13:11:31.433807+0000 mon.a (mon.0) 552 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi167.ojckkr", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:32.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[14758]: audit 2024-06-30T13:11:31.438442+0000 mon.a (mon.0) 553 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi167.ojckkr", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:32.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[14758]: audit 2024-06-30T13:11:31.444888+0000 mon.a (mon.0) 554 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:32.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[14758]: audit 2024-06-30T13:11:31.445956+0000 mon.a (mon.0) 555 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:32.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:32 smithi144 bash[14758]: cephadm 2024-06-30T13:11:31.446940+0000 mgr.y (mgr.14144) 303 : cephadm [INF] Deploying daemon rgw.foo.smithi167.ojckkr on smithi167 2024-06-30T13:11:32.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:32 smithi167 bash[14619]: audit 2024-06-30T13:11:31.432566+0000 mon.a (mon.0) 551 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:32.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:32 smithi167 bash[14619]: audit 2024-06-30T13:11:31.433807+0000 mon.a (mon.0) 552 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi167.ojckkr", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:32.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:32 smithi167 bash[14619]: audit 2024-06-30T13:11:31.438442+0000 mon.a (mon.0) 553 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.smithi167.ojckkr", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:32.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:32 smithi167 bash[14619]: audit 2024-06-30T13:11:31.444888+0000 mon.a (mon.0) 554 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:32.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:32 smithi167 bash[14619]: audit 2024-06-30T13:11:31.445956+0000 mon.a (mon.0) 555 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:32.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:32 smithi167 bash[14619]: cephadm 2024-06-30T13:11:31.446940+0000 mgr.y (mgr.14144) 303 : cephadm [INF] Deploying daemon rgw.foo.smithi167.ojckkr on smithi167 2024-06-30T13:11:33.407 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled rgw.smpl update... 2024-06-30T13:11:33.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:33 smithi167 bash[14619]: cluster 2024-06-30T13:11:32.429047+0000 mgr.y (mgr.14144) 304 : cluster [DBG] pgmap v280: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 5.8 KiB/s wr, 22 op/s 2024-06-30T13:11:33.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:33 smithi167 bash[14619]: audit 2024-06-30T13:11:33.403637+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:33.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:33 smithi144 bash[14758]: cluster 2024-06-30T13:11:32.429047+0000 mgr.y (mgr.14144) 304 : cluster [DBG] pgmap v280: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 5.8 KiB/s wr, 22 op/s 2024-06-30T13:11:33.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:33 smithi144 bash[14758]: audit 2024-06-30T13:11:33.403637+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:33.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:33 smithi144 bash[19242]: cluster 2024-06-30T13:11:32.429047+0000 mgr.y (mgr.14144) 304 : cluster [DBG] pgmap v280: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 2.8 KiB/s rd, 5.8 KiB/s wr, 22 op/s 2024-06-30T13:11:33.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:33 smithi144 bash[19242]: audit 2024-06-30T13:11:33.403637+0000 mon.a (mon.0) 556 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:34.251 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph osd pool create foo' 2024-06-30T13:11:34.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:34 smithi144 bash[14758]: audit 2024-06-30T13:11:33.396623+0000 mgr.y (mgr.14144) 305 : audit [DBG] from='client.14655 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:11:34.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:34 smithi144 bash[14758]: cephadm 2024-06-30T13:11:33.397207+0000 mgr.y (mgr.14144) 306 : cephadm [INF] Saving service rgw.smpl spec with placement count:2 2024-06-30T13:11:34.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:34 smithi144 bash[14758]: audit 2024-06-30T13:11:34.272394+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:34.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:34 smithi144 bash[14758]: audit 2024-06-30T13:11:34.274096+0000 mon.a (mon.0) 558 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:34.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:34 smithi144 bash[19242]: audit 2024-06-30T13:11:33.396623+0000 mgr.y (mgr.14144) 305 : audit [DBG] from='client.14655 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:11:34.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:34 smithi144 bash[19242]: cephadm 2024-06-30T13:11:33.397207+0000 mgr.y (mgr.14144) 306 : cephadm [INF] Saving service rgw.smpl spec with placement count:2 2024-06-30T13:11:34.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:34 smithi144 bash[19242]: audit 2024-06-30T13:11:34.272394+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:34.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:34 smithi144 bash[19242]: audit 2024-06-30T13:11:34.274096+0000 mon.a (mon.0) 558 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:34.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:34 smithi167 bash[14619]: audit 2024-06-30T13:11:33.396623+0000 mgr.y (mgr.14144) 305 : audit [DBG] from='client.14655 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "smpl", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:11:34.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:34 smithi167 bash[14619]: cephadm 2024-06-30T13:11:33.397207+0000 mgr.y (mgr.14144) 306 : cephadm [INF] Saving service rgw.smpl spec with placement count:2 2024-06-30T13:11:34.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:34 smithi167 bash[14619]: audit 2024-06-30T13:11:34.272394+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:34.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:34 smithi167 bash[14619]: audit 2024-06-30T13:11:34.274096+0000 mon.a (mon.0) 558 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:35.706 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:35 smithi144 bash[14758]: cluster 2024-06-30T13:11:34.429919+0000 mgr.y (mgr.14144) 307 : cluster [DBG] pgmap v281: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 5.1 KiB/s wr, 22 op/s 2024-06-30T13:11:35.706 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:35 smithi144 bash[19242]: cluster 2024-06-30T13:11:34.429919+0000 mgr.y (mgr.14144) 307 : cluster [DBG] pgmap v281: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 5.1 KiB/s wr, 22 op/s 2024-06-30T13:11:35.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:35 smithi167 bash[14619]: cluster 2024-06-30T13:11:34.429919+0000 mgr.y (mgr.14144) 307 : cluster [DBG] pgmap v281: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 4.1 KiB/s rd, 5.1 KiB/s wr, 22 op/s 2024-06-30T13:11:37.730 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:37 smithi144 bash[14758]: cluster 2024-06-30T13:11:36.431066+0000 mgr.y (mgr.14144) 308 : cluster [DBG] pgmap v282: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 2.3 KiB/s wr, 91 op/s 2024-06-30T13:11:37.730 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:37 smithi144 bash[19242]: cluster 2024-06-30T13:11:36.431066+0000 mgr.y (mgr.14144) 308 : cluster [DBG] pgmap v282: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 2.3 KiB/s wr, 91 op/s 2024-06-30T13:11:37.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:37 smithi167 bash[14619]: cluster 2024-06-30T13:11:36.431066+0000 mgr.y (mgr.14144) 308 : cluster [DBG] pgmap v282: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 2.3 KiB/s wr, 91 op/s 2024-06-30T13:11:39.092 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'foo' created 2024-06-30T13:11:39.414 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:39 smithi167 bash[14619]: audit 2024-06-30T13:11:38.060761+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:39.414 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:39 smithi167 bash[14619]: audit 2024-06-30T13:11:38.140122+0000 mon.c (mon.2) 22 : audit [INF] from='client.? 172.21.15.144:0/697892589' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-30T13:11:39.414 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:39 smithi167 bash[14619]: audit 2024-06-30T13:11:38.140741+0000 mon.a (mon.0) 560 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-30T13:11:39.415 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:39 smithi167 bash[14619]: cluster 2024-06-30T13:11:38.432037+0000 mgr.y (mgr.14144) 309 : cluster [DBG] pgmap v283: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s rd, 511 B/s wr, 174 op/s 2024-06-30T13:11:39.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:39 smithi144 bash[14758]: audit 2024-06-30T13:11:38.060761+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:39.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:39 smithi144 bash[14758]: audit 2024-06-30T13:11:38.140122+0000 mon.c (mon.2) 22 : audit [INF] from='client.? 172.21.15.144:0/697892589' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-30T13:11:39.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:39 smithi144 bash[14758]: audit 2024-06-30T13:11:38.140741+0000 mon.a (mon.0) 560 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-30T13:11:39.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:39 smithi144 bash[14758]: cluster 2024-06-30T13:11:38.432037+0000 mgr.y (mgr.14144) 309 : cluster [DBG] pgmap v283: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s rd, 511 B/s wr, 174 op/s 2024-06-30T13:11:39.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:39 smithi144 bash[19242]: audit 2024-06-30T13:11:38.060761+0000 mon.a (mon.0) 559 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:39.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:39 smithi144 bash[19242]: audit 2024-06-30T13:11:38.140122+0000 mon.c (mon.2) 22 : audit [INF] from='client.? 172.21.15.144:0/697892589' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-30T13:11:39.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:39 smithi144 bash[19242]: audit 2024-06-30T13:11:38.140741+0000 mon.a (mon.0) 560 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "foo"}]: dispatch 2024-06-30T13:11:39.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:39 smithi144 bash[19242]: cluster 2024-06-30T13:11:38.432037+0000 mgr.y (mgr.14144) 309 : cluster [DBG] pgmap v283: 129 pgs: 129 active+clean; 27 KiB data, 49 MiB used, 715 GiB / 715 GiB avail; 87 KiB/s rd, 511 B/s wr, 174 op/s 2024-06-30T13:11:39.817 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'rbd pool init foo' 2024-06-30T13:11:40.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: audit 2024-06-30T13:11:39.075594+0000 mon.a (mon.0) 561 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-06-30T13:11:40.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: cluster 2024-06-30T13:11:39.075827+0000 mon.a (mon.0) 562 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-06-30T13:11:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: audit 2024-06-30T13:11:39.085343+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: audit 2024-06-30T13:11:39.095806+0000 mon.a (mon.0) 564 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: cephadm 2024-06-30T13:11:39.100201+0000 mgr.y (mgr.14144) 310 : cephadm [INF] Saving service rgw.smpl spec with placement count:2 2024-06-30T13:11:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: audit 2024-06-30T13:11:39.104090+0000 mon.a (mon.0) 565 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: audit 2024-06-30T13:11:39.105312+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi167.onkqne", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: audit 2024-06-30T13:11:39.110124+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi167.onkqne", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: audit 2024-06-30T13:11:39.117913+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: audit 2024-06-30T13:11:39.119076+0000 mon.a (mon.0) 569 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[14758]: cephadm 2024-06-30T13:11:39.120071+0000 mgr.y (mgr.14144) 311 : cephadm [INF] Deploying daemon rgw.smpl.smithi167.onkqne on smithi167 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: audit 2024-06-30T13:11:39.075594+0000 mon.a (mon.0) 561 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: cluster 2024-06-30T13:11:39.075827+0000 mon.a (mon.0) 562 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: audit 2024-06-30T13:11:39.085343+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: audit 2024-06-30T13:11:39.095806+0000 mon.a (mon.0) 564 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: cephadm 2024-06-30T13:11:39.100201+0000 mgr.y (mgr.14144) 310 : cephadm [INF] Saving service rgw.smpl spec with placement count:2 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: audit 2024-06-30T13:11:39.104090+0000 mon.a (mon.0) 565 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: audit 2024-06-30T13:11:39.105312+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi167.onkqne", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: audit 2024-06-30T13:11:39.110124+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi167.onkqne", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: audit 2024-06-30T13:11:39.117913+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: audit 2024-06-30T13:11:39.119076+0000 mon.a (mon.0) 569 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:40 smithi144 bash[19242]: cephadm 2024-06-30T13:11:39.120071+0000 mgr.y (mgr.14144) 311 : cephadm [INF] Deploying daemon rgw.smpl.smithi167.onkqne on smithi167 2024-06-30T13:11:40.519 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: audit 2024-06-30T13:11:39.075594+0000 mon.a (mon.0) 561 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "foo"}]': finished 2024-06-30T13:11:40.519 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: cluster 2024-06-30T13:11:39.075827+0000 mon.a (mon.0) 562 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-06-30T13:11:40.519 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: audit 2024-06-30T13:11:39.085343+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.519 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: audit 2024-06-30T13:11:39.095806+0000 mon.a (mon.0) 564 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.519 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: cephadm 2024-06-30T13:11:39.100201+0000 mgr.y (mgr.14144) 310 : cephadm [INF] Saving service rgw.smpl spec with placement count:2 2024-06-30T13:11:40.519 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: audit 2024-06-30T13:11:39.104090+0000 mon.a (mon.0) 565 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.519 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: audit 2024-06-30T13:11:39.105312+0000 mon.a (mon.0) 566 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi167.onkqne", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:40.519 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: audit 2024-06-30T13:11:39.110124+0000 mon.a (mon.0) 567 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi167.onkqne", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:40.520 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: audit 2024-06-30T13:11:39.117913+0000 mon.a (mon.0) 568 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:40.520 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: audit 2024-06-30T13:11:39.119076+0000 mon.a (mon.0) 569 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:40.520 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:40 smithi167 bash[14619]: cephadm 2024-06-30T13:11:39.120071+0000 mgr.y (mgr.14144) 311 : cephadm [INF] Deploying daemon rgw.smpl.smithi167.onkqne on smithi167 2024-06-30T13:11:41.372 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:41 smithi167 bash[14619]: cluster 2024-06-30T13:11:40.080390+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-06-30T13:11:41.372 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:41 smithi167 bash[14619]: cluster 2024-06-30T13:11:40.432776+0000 mgr.y (mgr.14144) 312 : cluster [DBG] pgmap v286: 161 pgs: 5 creating+peering, 27 unknown, 129 active+clean; 27 KiB data, 55 MiB used, 715 GiB / 715 GiB avail; 208 KiB/s rd, 127 B/s wr, 404 op/s 2024-06-30T13:11:41.372 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:41 smithi167 bash[14619]: cluster 2024-06-30T13:11:41.082568+0000 mon.a (mon.0) 571 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-06-30T13:11:41.374 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:41 smithi144 bash[14758]: cluster 2024-06-30T13:11:40.080390+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-06-30T13:11:41.374 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:41 smithi144 bash[14758]: cluster 2024-06-30T13:11:40.432776+0000 mgr.y (mgr.14144) 312 : cluster [DBG] pgmap v286: 161 pgs: 5 creating+peering, 27 unknown, 129 active+clean; 27 KiB data, 55 MiB used, 715 GiB / 715 GiB avail; 208 KiB/s rd, 127 B/s wr, 404 op/s 2024-06-30T13:11:41.374 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:41 smithi144 bash[14758]: cluster 2024-06-30T13:11:41.082568+0000 mon.a (mon.0) 571 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-06-30T13:11:41.374 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:41 smithi144 bash[19242]: cluster 2024-06-30T13:11:40.080390+0000 mon.a (mon.0) 570 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-06-30T13:11:41.374 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:41 smithi144 bash[19242]: cluster 2024-06-30T13:11:40.432776+0000 mgr.y (mgr.14144) 312 : cluster [DBG] pgmap v286: 161 pgs: 5 creating+peering, 27 unknown, 129 active+clean; 27 KiB data, 55 MiB used, 715 GiB / 715 GiB avail; 208 KiB/s rd, 127 B/s wr, 404 op/s 2024-06-30T13:11:41.374 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:41 smithi144 bash[19242]: cluster 2024-06-30T13:11:41.082568+0000 mon.a (mon.0) 571 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-06-30T13:11:43.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[14758]: audit 2024-06-30T13:11:41.879098+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:43.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[14758]: audit 2024-06-30T13:11:41.881414+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi144.rfqrlh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:43.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[14758]: audit 2024-06-30T13:11:41.886892+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi144.rfqrlh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:43.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[14758]: audit 2024-06-30T13:11:41.893951+0000 mon.a (mon.0) 575 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:43.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[14758]: audit 2024-06-30T13:11:41.895359+0000 mon.a (mon.0) 576 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:43.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[14758]: cephadm 2024-06-30T13:11:41.896968+0000 mgr.y (mgr.14144) 313 : cephadm [INF] Deploying daemon rgw.smpl.smithi144.rfqrlh on smithi144 2024-06-30T13:11:43.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[14758]: cluster 2024-06-30T13:11:42.433718+0000 mgr.y (mgr.14144) 314 : cluster [DBG] pgmap v288: 161 pgs: 5 creating+peering, 13 unknown, 143 active+clean; 27 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 273 KiB/s rd, 170 B/s wr, 507 op/s 2024-06-30T13:11:43.262 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[19242]: audit 2024-06-30T13:11:41.879098+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:43.262 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[19242]: audit 2024-06-30T13:11:41.881414+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi144.rfqrlh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:43.262 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[19242]: audit 2024-06-30T13:11:41.886892+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi144.rfqrlh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:43.262 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[19242]: audit 2024-06-30T13:11:41.893951+0000 mon.a (mon.0) 575 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:43.262 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[19242]: audit 2024-06-30T13:11:41.895359+0000 mon.a (mon.0) 576 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:43.262 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[19242]: cephadm 2024-06-30T13:11:41.896968+0000 mgr.y (mgr.14144) 313 : cephadm [INF] Deploying daemon rgw.smpl.smithi144.rfqrlh on smithi144 2024-06-30T13:11:43.263 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:42 smithi144 bash[19242]: cluster 2024-06-30T13:11:42.433718+0000 mgr.y (mgr.14144) 314 : cluster [DBG] pgmap v288: 161 pgs: 5 creating+peering, 13 unknown, 143 active+clean; 27 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 273 KiB/s rd, 170 B/s wr, 507 op/s 2024-06-30T13:11:43.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:42 smithi167 bash[14619]: audit 2024-06-30T13:11:41.879098+0000 mon.a (mon.0) 572 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:43.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:42 smithi167 bash[14619]: audit 2024-06-30T13:11:41.881414+0000 mon.a (mon.0) 573 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi144.rfqrlh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-06-30T13:11:43.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:42 smithi167 bash[14619]: audit 2024-06-30T13:11:41.886892+0000 mon.a (mon.0) 574 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.smpl.smithi144.rfqrlh", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-06-30T13:11:43.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:42 smithi167 bash[14619]: audit 2024-06-30T13:11:41.893951+0000 mon.a (mon.0) 575 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:43.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:42 smithi167 bash[14619]: audit 2024-06-30T13:11:41.895359+0000 mon.a (mon.0) 576 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:43.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:42 smithi167 bash[14619]: cephadm 2024-06-30T13:11:41.896968+0000 mgr.y (mgr.14144) 313 : cephadm [INF] Deploying daemon rgw.smpl.smithi144.rfqrlh on smithi144 2024-06-30T13:11:43.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:42 smithi167 bash[14619]: cluster 2024-06-30T13:11:42.433718+0000 mgr.y (mgr.14144) 314 : cluster [DBG] pgmap v288: 161 pgs: 5 creating+peering, 13 unknown, 143 active+clean; 27 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 273 KiB/s rd, 170 B/s wr, 507 op/s 2024-06-30T13:11:44.509 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:44 smithi144 bash[14758]: audit 2024-06-30T13:11:44.326397+0000 mon.a (mon.0) 577 : audit [INF] from='client.? 172.21.15.144:0/724951203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-06-30T13:11:44.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:44 smithi144 bash[19242]: audit 2024-06-30T13:11:44.326397+0000 mon.a (mon.0) 577 : audit [INF] from='client.? 172.21.15.144:0/724951203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-06-30T13:11:44.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:44 smithi167 bash[14619]: audit 2024-06-30T13:11:44.326397+0000 mon.a (mon.0) 577 : audit [INF] from='client.? 172.21.15.144:0/724951203' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]: dispatch 2024-06-30T13:11:45.455 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:45 smithi144 bash[14758]: audit 2024-06-30T13:11:44.384220+0000 mon.a (mon.0) 578 : audit [INF] from='client.? 172.21.15.144:0/724951203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-06-30T13:11:45.455 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:45 smithi144 bash[14758]: cluster 2024-06-30T13:11:44.385315+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-06-30T13:11:45.455 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:45 smithi144 bash[14758]: cluster 2024-06-30T13:11:44.434466+0000 mgr.y (mgr.14144) 315 : cluster [DBG] pgmap v290: 161 pgs: 5 creating+peering, 156 active+clean; 27 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 141 KiB/s rd, 190 B/s wr, 247 op/s 2024-06-30T13:11:45.456 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:45 smithi144 bash[19242]: audit 2024-06-30T13:11:44.384220+0000 mon.a (mon.0) 578 : audit [INF] from='client.? 172.21.15.144:0/724951203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-06-30T13:11:45.456 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:45 smithi144 bash[19242]: cluster 2024-06-30T13:11:44.385315+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-06-30T13:11:45.456 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:45 smithi144 bash[19242]: cluster 2024-06-30T13:11:44.434466+0000 mgr.y (mgr.14144) 315 : cluster [DBG] pgmap v290: 161 pgs: 5 creating+peering, 156 active+clean; 27 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 141 KiB/s rd, 190 B/s wr, 247 op/s 2024-06-30T13:11:45.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:45 smithi167 bash[14619]: audit 2024-06-30T13:11:44.384220+0000 mon.a (mon.0) 578 : audit [INF] from='client.? 172.21.15.144:0/724951203' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "foo","app": "rbd"}]': finished 2024-06-30T13:11:45.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:45 smithi167 bash[14619]: cluster 2024-06-30T13:11:44.385315+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-06-30T13:11:45.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:45 smithi167 bash[14619]: cluster 2024-06-30T13:11:44.434466+0000 mgr.y (mgr.14144) 315 : cluster [DBG] pgmap v290: 161 pgs: 5 creating+peering, 156 active+clean; 27 KiB data, 60 MiB used, 715 GiB / 715 GiB avail; 141 KiB/s rd, 190 B/s wr, 247 op/s 2024-06-30T13:11:46.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:46 smithi144 bash[14758]: cluster 2024-06-30T13:11:45.402251+0000 mon.a (mon.0) 580 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-06-30T13:11:46.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:46 smithi144 bash[14758]: audit 2024-06-30T13:11:45.705261+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:46.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:46 smithi144 bash[14758]: audit 2024-06-30T13:11:45.706401+0000 mon.a (mon.0) 582 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:46 smithi144 bash[19242]: cluster 2024-06-30T13:11:45.402251+0000 mon.a (mon.0) 580 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-06-30T13:11:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:46 smithi144 bash[19242]: audit 2024-06-30T13:11:45.705261+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:46.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:46 smithi144 bash[19242]: audit 2024-06-30T13:11:45.706401+0000 mon.a (mon.0) 582 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:46 smithi167 bash[14619]: cluster 2024-06-30T13:11:45.402251+0000 mon.a (mon.0) 580 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-06-30T13:11:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:46 smithi167 bash[14619]: audit 2024-06-30T13:11:45.705261+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:46.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:46 smithi167 bash[14619]: audit 2024-06-30T13:11:45.706401+0000 mon.a (mon.0) 582 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:47.613 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph orch apply iscsi foo u p' 2024-06-30T13:11:47.759 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:47 smithi144 bash[19242]: cluster 2024-06-30T13:11:46.407858+0000 mon.a (mon.0) 583 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-06-30T13:11:47.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:47 smithi144 bash[19242]: cluster 2024-06-30T13:11:46.435078+0000 mgr.y (mgr.14144) 316 : cluster [DBG] pgmap v293: 161 pgs: 161 active+clean; 27 KiB data, 65 MiB used, 715 GiB / 715 GiB avail; 129 KiB/s rd, 191 B/s wr, 223 op/s 2024-06-30T13:11:47.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:47 smithi144 bash[14758]: cluster 2024-06-30T13:11:46.407858+0000 mon.a (mon.0) 583 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-06-30T13:11:47.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:47 smithi144 bash[14758]: cluster 2024-06-30T13:11:46.435078+0000 mgr.y (mgr.14144) 316 : cluster [DBG] pgmap v293: 161 pgs: 161 active+clean; 27 KiB data, 65 MiB used, 715 GiB / 715 GiB avail; 129 KiB/s rd, 191 B/s wr, 223 op/s 2024-06-30T13:11:47.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:47 smithi167 bash[14619]: cluster 2024-06-30T13:11:46.407858+0000 mon.a (mon.0) 583 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-06-30T13:11:47.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:47 smithi167 bash[14619]: cluster 2024-06-30T13:11:46.435078+0000 mgr.y (mgr.14144) 316 : cluster [DBG] pgmap v293: 161 pgs: 161 active+clean; 27 KiB data, 65 MiB used, 715 GiB / 715 GiB avail; 129 KiB/s rd, 191 B/s wr, 223 op/s 2024-06-30T13:11:49.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:48 smithi144 bash[14758]: cluster 2024-06-30T13:11:48.435846+0000 mgr.y (mgr.14144) 317 : cluster [DBG] pgmap v294: 161 pgs: 161 active+clean; 27 KiB data, 73 MiB used, 715 GiB / 715 GiB avail; 157 KiB/s rd, 170 B/s wr, 276 op/s 2024-06-30T13:11:49.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:48 smithi144 bash[19242]: cluster 2024-06-30T13:11:48.435846+0000 mgr.y (mgr.14144) 317 : cluster [DBG] pgmap v294: 161 pgs: 161 active+clean; 27 KiB data, 73 MiB used, 715 GiB / 715 GiB avail; 157 KiB/s rd, 170 B/s wr, 276 op/s 2024-06-30T13:11:49.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:48 smithi167 bash[14619]: cluster 2024-06-30T13:11:48.435846+0000 mgr.y (mgr.14144) 317 : cluster [DBG] pgmap v294: 161 pgs: 161 active+clean; 27 KiB data, 73 MiB used, 715 GiB / 715 GiB avail; 157 KiB/s rd, 170 B/s wr, 276 op/s 2024-06-30T13:11:50.846 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:50 smithi144 bash[14758]: audit 2024-06-30T13:11:49.555342+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:50.846 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:50 smithi144 bash[14758]: cluster 2024-06-30T13:11:50.437243+0000 mgr.y (mgr.14144) 318 : cluster [DBG] pgmap v295: 161 pgs: 161 active+clean; 27 KiB data, 82 MiB used, 715 GiB / 715 GiB avail; 237 KiB/s rd, 507 B/s wr, 415 op/s 2024-06-30T13:11:50.846 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:50 smithi144 bash[19242]: audit 2024-06-30T13:11:49.555342+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:50.847 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:50 smithi144 bash[19242]: cluster 2024-06-30T13:11:50.437243+0000 mgr.y (mgr.14144) 318 : cluster [DBG] pgmap v295: 161 pgs: 161 active+clean; 27 KiB data, 82 MiB used, 715 GiB / 715 GiB avail; 237 KiB/s rd, 507 B/s wr, 415 op/s 2024-06-30T13:11:51.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:50 smithi167 bash[14619]: audit 2024-06-30T13:11:49.555342+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:51.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:50 smithi167 bash[14619]: cluster 2024-06-30T13:11:50.437243+0000 mgr.y (mgr.14144) 318 : cluster [DBG] pgmap v295: 161 pgs: 161 active+clean; 27 KiB data, 82 MiB used, 715 GiB / 715 GiB avail; 237 KiB/s rd, 507 B/s wr, 415 op/s 2024-06-30T13:11:51.804 INFO:teuthology.orchestra.run.smithi144.stdout:Scheduled iscsi.foo update... 2024-06-30T13:11:52.536 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'sleep 120' 2024-06-30T13:11:52.797 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: audit 2024-06-30T13:11:51.704669+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:52.797 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: audit 2024-06-30T13:11:51.795979+0000 mgr.y (mgr.14144) 319 : audit [DBG] from='client.24677 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:11:52.797 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: cephadm 2024-06-30T13:11:51.796520+0000 mgr.y (mgr.14144) 320 : cephadm [INF] Saving service iscsi.foo spec with placement count:1 2024-06-30T13:11:52.797 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: audit 2024-06-30T13:11:51.800959+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:52.797 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: audit 2024-06-30T13:11:51.959401+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:52.797 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: cephadm 2024-06-30T13:11:51.965495+0000 mgr.y (mgr.14144) 321 : cephadm [INF] Checking pool "foo" exists for service iscsi.foo 2024-06-30T13:11:52.797 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: audit 2024-06-30T13:11:51.975363+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi167.teebbw", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-30T13:11:52.797 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: audit 2024-06-30T13:11:51.981744+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi167.teebbw", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: audit 2024-06-30T13:11:51.991952+0000 mon.a (mon.0) 590 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: cephadm 2024-06-30T13:11:51.993310+0000 mgr.y (mgr.14144) 322 : cephadm [INF] Deploying daemon iscsi.foo.smithi167.teebbw on smithi167 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[14758]: cluster 2024-06-30T13:11:52.438492+0000 mgr.y (mgr.14144) 323 : cluster [DBG] pgmap v296: 161 pgs: 161 active+clean; 27 KiB data, 83 MiB used, 715 GiB / 715 GiB avail; 246 KiB/s rd, 511 B/s wr, 430 op/s 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: audit 2024-06-30T13:11:51.704669+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: audit 2024-06-30T13:11:51.795979+0000 mgr.y (mgr.14144) 319 : audit [DBG] from='client.24677 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: cephadm 2024-06-30T13:11:51.796520+0000 mgr.y (mgr.14144) 320 : cephadm [INF] Saving service iscsi.foo spec with placement count:1 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: audit 2024-06-30T13:11:51.800959+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: audit 2024-06-30T13:11:51.959401+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: cephadm 2024-06-30T13:11:51.965495+0000 mgr.y (mgr.14144) 321 : cephadm [INF] Checking pool "foo" exists for service iscsi.foo 2024-06-30T13:11:52.798 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: audit 2024-06-30T13:11:51.975363+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi167.teebbw", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-30T13:11:52.799 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: audit 2024-06-30T13:11:51.981744+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi167.teebbw", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-30T13:11:52.799 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: audit 2024-06-30T13:11:51.991952+0000 mon.a (mon.0) 590 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:52.799 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: cephadm 2024-06-30T13:11:51.993310+0000 mgr.y (mgr.14144) 322 : cephadm [INF] Deploying daemon iscsi.foo.smithi167.teebbw on smithi167 2024-06-30T13:11:52.799 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:52 smithi144 bash[19242]: cluster 2024-06-30T13:11:52.438492+0000 mgr.y (mgr.14144) 323 : cluster [DBG] pgmap v296: 161 pgs: 161 active+clean; 27 KiB data, 83 MiB used, 715 GiB / 715 GiB avail; 246 KiB/s rd, 511 B/s wr, 430 op/s 2024-06-30T13:11:53.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: audit 2024-06-30T13:11:51.704669+0000 mon.a (mon.0) 585 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: audit 2024-06-30T13:11:51.795979+0000 mgr.y (mgr.14144) 319 : audit [DBG] from='client.24677 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "foo", "api_user": "u", "api_password": "p", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: cephadm 2024-06-30T13:11:51.796520+0000 mgr.y (mgr.14144) 320 : cephadm [INF] Saving service iscsi.foo spec with placement count:1 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: audit 2024-06-30T13:11:51.800959+0000 mon.a (mon.0) 586 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: audit 2024-06-30T13:11:51.959401+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: cephadm 2024-06-30T13:11:51.965495+0000 mgr.y (mgr.14144) 321 : cephadm [INF] Checking pool "foo" exists for service iscsi.foo 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: audit 2024-06-30T13:11:51.975363+0000 mon.a (mon.0) 588 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi167.teebbw", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: audit 2024-06-30T13:11:51.981744+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.foo.smithi167.teebbw", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: audit 2024-06-30T13:11:51.991952+0000 mon.a (mon.0) 590 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: cephadm 2024-06-30T13:11:51.993310+0000 mgr.y (mgr.14144) 322 : cephadm [INF] Deploying daemon iscsi.foo.smithi167.teebbw on smithi167 2024-06-30T13:11:53.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:52 smithi167 bash[14619]: cluster 2024-06-30T13:11:52.438492+0000 mgr.y (mgr.14144) 323 : cluster [DBG] pgmap v296: 161 pgs: 161 active+clean; 27 KiB data, 83 MiB used, 715 GiB / 715 GiB avail; 246 KiB/s rd, 511 B/s wr, 430 op/s 2024-06-30T13:11:54.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:53 smithi144 bash[14758]: cluster 2024-06-30T13:11:52.998008+0000 mon.a (mon.0) 591 : cluster [DBG] mgrmap e15: y(active, since 7m), standbys: x 2024-06-30T13:11:54.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:53 smithi144 bash[19242]: cluster 2024-06-30T13:11:52.998008+0000 mon.a (mon.0) 591 : cluster [DBG] mgrmap e15: y(active, since 7m), standbys: x 2024-06-30T13:11:54.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:53 smithi167 bash[14619]: cluster 2024-06-30T13:11:52.998008+0000 mon.a (mon.0) 591 : cluster [DBG] mgrmap e15: y(active, since 7m), standbys: x 2024-06-30T13:11:55.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:54 smithi144 bash[14758]: cluster 2024-06-30T13:11:54.439689+0000 mgr.y (mgr.14144) 324 : cluster [DBG] pgmap v297: 161 pgs: 161 active+clean; 27 KiB data, 87 MiB used, 715 GiB / 715 GiB avail; 233 KiB/s rd, 453 B/s wr, 406 op/s 2024-06-30T13:11:55.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:54 smithi144 bash[14758]: cluster 2024-06-30T13:11:54.698558+0000 mon.a (mon.0) 592 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-06-30T13:11:55.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:54 smithi144 bash[14758]: audit 2024-06-30T13:11:54.987027+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:55.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:54 smithi144 bash[14758]: audit 2024-06-30T13:11:54.988846+0000 mon.a (mon.0) 594 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:55.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:55 smithi144 bash[19242]: cluster 2024-06-30T13:11:54.439689+0000 mgr.y (mgr.14144) 324 : cluster [DBG] pgmap v297: 161 pgs: 161 active+clean; 27 KiB data, 87 MiB used, 715 GiB / 715 GiB avail; 233 KiB/s rd, 453 B/s wr, 406 op/s 2024-06-30T13:11:55.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:55 smithi144 bash[19242]: cluster 2024-06-30T13:11:54.698558+0000 mon.a (mon.0) 592 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-06-30T13:11:55.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:55 smithi144 bash[19242]: audit 2024-06-30T13:11:54.987027+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:55.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:55 smithi144 bash[19242]: audit 2024-06-30T13:11:54.988846+0000 mon.a (mon.0) 594 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:55.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:55 smithi167 bash[14619]: cluster 2024-06-30T13:11:54.439689+0000 mgr.y (mgr.14144) 324 : cluster [DBG] pgmap v297: 161 pgs: 161 active+clean; 27 KiB data, 87 MiB used, 715 GiB / 715 GiB avail; 233 KiB/s rd, 453 B/s wr, 406 op/s 2024-06-30T13:11:55.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:55 smithi167 bash[14619]: cluster 2024-06-30T13:11:54.698558+0000 mon.a (mon.0) 592 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-06-30T13:11:55.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:55 smithi167 bash[14619]: audit 2024-06-30T13:11:54.987027+0000 mon.a (mon.0) 593 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:11:55.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:55 smithi167 bash[14619]: audit 2024-06-30T13:11:54.988846+0000 mon.a (mon.0) 594 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:11:56.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:56 smithi167 bash[14619]: audit 2024-06-30T13:11:56.333308+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2024-06-30T13:11:56.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:56 smithi167 bash[14619]: audit 2024-06-30T13:11:56.333976+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-06-30T13:11:56.670 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:56 smithi167 bash[14619]: audit 2024-06-30T13:11:56.334425+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-06-30T13:11:56.671 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:56 smithi167 bash[14619]: audit 2024-06-30T13:11:56.334864+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-06-30T13:11:56.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:56 smithi144 bash[14758]: audit 2024-06-30T13:11:56.333308+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2024-06-30T13:11:56.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:56 smithi144 bash[14758]: audit 2024-06-30T13:11:56.333976+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-06-30T13:11:56.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:56 smithi144 bash[14758]: audit 2024-06-30T13:11:56.334425+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-06-30T13:11:56.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:56 smithi144 bash[14758]: audit 2024-06-30T13:11:56.334864+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-06-30T13:11:56.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:56 smithi144 bash[19242]: audit 2024-06-30T13:11:56.333308+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]: dispatch 2024-06-30T13:11:56.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:56 smithi144 bash[19242]: audit 2024-06-30T13:11:56.333976+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]: dispatch 2024-06-30T13:11:56.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:56 smithi144 bash[19242]: audit 2024-06-30T13:11:56.334425+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-06-30T13:11:56.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:56 smithi144 bash[19242]: audit 2024-06-30T13:11:56.334864+0000 mon.a (mon.0) 598 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-06-30T13:11:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: audit 2024-06-30T13:11:56.391951+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]': finished 2024-06-30T13:11:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: audit 2024-06-30T13:11:56.392168+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-06-30T13:11:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: audit 2024-06-30T13:11:56.392374+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-06-30T13:11:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: audit 2024-06-30T13:11:56.392574+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-06-30T13:11:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: cluster 2024-06-30T13:11:56.392755+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-06-30T13:11:57.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: cluster 2024-06-30T13:11:56.440476+0000 mgr.y (mgr.14144) 325 : cluster [DBG] pgmap v300: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 173 KiB/s rd, 383 B/s wr, 299 op/s 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: audit 2024-06-30T13:11:56.523969+0000 mon.a (mon.0) 604 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: audit 2024-06-30T13:11:56.524250+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: audit 2024-06-30T13:11:57.312395+0000 mon.b (mon.1) 24 : audit [DBG] from='client.? 172.21.15.167:0/2010141621' entity='client.iscsi.foo.smithi167.teebbw' cmd=[{"prefix": "osd blacklist ls"}]: dispatch 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[19242]: cluster 2024-06-30T13:11:57.395897+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: audit 2024-06-30T13:11:56.391951+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]': finished 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: audit 2024-06-30T13:11:56.392168+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: audit 2024-06-30T13:11:56.392374+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: audit 2024-06-30T13:11:56.392574+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: cluster 2024-06-30T13:11:56.392755+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: cluster 2024-06-30T13:11:56.440476+0000 mgr.y (mgr.14144) 325 : cluster [DBG] pgmap v300: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 173 KiB/s rd, 383 B/s wr, 299 op/s 2024-06-30T13:11:57.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: audit 2024-06-30T13:11:56.523969+0000 mon.a (mon.0) 604 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:11:57.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: audit 2024-06-30T13:11:56.524250+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:11:57.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: audit 2024-06-30T13:11:57.312395+0000 mon.b (mon.1) 24 : audit [DBG] from='client.? 172.21.15.167:0/2010141621' entity='client.iscsi.foo.smithi167.teebbw' cmd=[{"prefix": "osd blacklist ls"}]: dispatch 2024-06-30T13:11:57.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:57 smithi144 bash[14758]: cluster 2024-06-30T13:11:57.395897+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-06-30T13:11:57.777 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: audit 2024-06-30T13:11:56.391951+0000 mon.a (mon.0) 599 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [7, 2]}]': finished 2024-06-30T13:11:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: audit 2024-06-30T13:11:56.392168+0000 mon.a (mon.0) 600 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 2]}]': finished 2024-06-30T13:11:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: audit 2024-06-30T13:11:56.392374+0000 mon.a (mon.0) 601 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-06-30T13:11:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: audit 2024-06-30T13:11:56.392574+0000 mon.a (mon.0) 602 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-06-30T13:11:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: cluster 2024-06-30T13:11:56.392755+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-06-30T13:11:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: cluster 2024-06-30T13:11:56.440476+0000 mgr.y (mgr.14144) 325 : cluster [DBG] pgmap v300: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 173 KiB/s rd, 383 B/s wr, 299 op/s 2024-06-30T13:11:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: audit 2024-06-30T13:11:56.523969+0000 mon.a (mon.0) 604 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:11:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: audit 2024-06-30T13:11:56.524250+0000 mon.a (mon.0) 605 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:11:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: audit 2024-06-30T13:11:57.312395+0000 mon.b (mon.1) 24 : audit [DBG] from='client.? 172.21.15.167:0/2010141621' entity='client.iscsi.foo.smithi167.teebbw' cmd=[{"prefix": "osd blacklist ls"}]: dispatch 2024-06-30T13:11:57.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:57 smithi167 bash[14619]: cluster 2024-06-30T13:11:57.395897+0000 mon.a (mon.0) 606 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-06-30T13:11:59.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:11:58 smithi144 bash[19242]: cluster 2024-06-30T13:11:58.441439+0000 mgr.y (mgr.14144) 326 : cluster [DBG] pgmap v302: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 92 op/s 2024-06-30T13:11:59.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:11:58 smithi144 bash[14758]: cluster 2024-06-30T13:11:58.441439+0000 mgr.y (mgr.14144) 326 : cluster [DBG] pgmap v302: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 92 op/s 2024-06-30T13:11:59.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:11:58 smithi167 bash[14619]: cluster 2024-06-30T13:11:58.441439+0000 mgr.y (mgr.14144) 326 : cluster [DBG] pgmap v302: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 0 B/s wr, 92 op/s 2024-06-30T13:12:01.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.094121+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.101947+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.110273+0000 mon.a (mon.0) 609 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.111088+0000 mgr.y (mgr.14144) 327 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: cephadm 2024-06-30T13:12:00.114646+0000 mgr.y (mgr.14144) 328 : cephadm [INF] Adding iSCSI gateway http://:@172.21.15.167:5000 to Dashboard 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.115037+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.115563+0000 mgr.y (mgr.14144) 329 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.125056+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.128618+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi167"}]: dispatch 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.129851+0000 mgr.y (mgr.14144) 330 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi167"}]: dispatch 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.146045+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.148030+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: audit 2024-06-30T13:12:00.156797+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[14758]: cluster 2024-06-30T13:12:00.442629+0000 mgr.y (mgr.14144) 331 : cluster [DBG] pgmap v303: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.094121+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.101947+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.110273+0000 mon.a (mon.0) 609 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.111088+0000 mgr.y (mgr.14144) 327 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: cephadm 2024-06-30T13:12:00.114646+0000 mgr.y (mgr.14144) 328 : cephadm [INF] Adding iSCSI gateway http://:@172.21.15.167:5000 to Dashboard 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.115037+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.115563+0000 mgr.y (mgr.14144) 329 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.125056+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.128618+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi167"}]: dispatch 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.129851+0000 mgr.y (mgr.14144) 330 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi167"}]: dispatch 2024-06-30T13:12:01.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.146045+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.513 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.148030+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:12:01.513 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: audit 2024-06-30T13:12:00.156797+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.513 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:01 smithi144 bash[19242]: cluster 2024-06-30T13:12:00.442629+0000 mgr.y (mgr.14144) 331 : cluster [DBG] pgmap v303: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:01.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.094121+0000 mon.a (mon.0) 607 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.101947+0000 mon.a (mon.0) 608 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.110273+0000 mon.a (mon.0) 609 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.111088+0000 mgr.y (mgr.14144) 327 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: cephadm 2024-06-30T13:12:00.114646+0000 mgr.y (mgr.14144) 328 : cephadm [INF] Adding iSCSI gateway http://:@172.21.15.167:5000 to Dashboard 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.115037+0000 mon.a (mon.0) 610 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.115563+0000 mgr.y (mgr.14144) 329 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.125056+0000 mon.a (mon.0) 611 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.128618+0000 mon.a (mon.0) 612 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi167"}]: dispatch 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.129851+0000 mgr.y (mgr.14144) 330 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi167"}]: dispatch 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.146045+0000 mon.a (mon.0) 613 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.148030+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: audit 2024-06-30T13:12:00.156797+0000 mon.a (mon.0) 615 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:12:01.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:01 smithi167 bash[14619]: cluster 2024-06-30T13:12:00.442629+0000 mgr.y (mgr.14144) 331 : cluster [DBG] pgmap v303: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:03.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:02 smithi144 bash[14758]: cluster 2024-06-30T13:12:02.443809+0000 mgr.y (mgr.14144) 332 : cluster [DBG] pgmap v304: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 528 B/s rd, 132 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:03.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:02 smithi144 bash[19242]: cluster 2024-06-30T13:12:02.443809+0000 mgr.y (mgr.14144) 332 : cluster [DBG] pgmap v304: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 528 B/s rd, 132 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:03.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:02 smithi167 bash[14619]: cluster 2024-06-30T13:12:02.443809+0000 mgr.y (mgr.14144) 332 : cluster [DBG] pgmap v304: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 528 B/s rd, 132 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:05.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:04 smithi144 bash[14758]: cluster 2024-06-30T13:12:04.444941+0000 mgr.y (mgr.14144) 333 : cluster [DBG] pgmap v305: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 508 B/s rd, 127 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:05.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:04 smithi144 bash[19242]: cluster 2024-06-30T13:12:04.444941+0000 mgr.y (mgr.14144) 333 : cluster [DBG] pgmap v305: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 508 B/s rd, 127 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:05.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:04 smithi167 bash[14619]: cluster 2024-06-30T13:12:04.444941+0000 mgr.y (mgr.14144) 333 : cluster [DBG] pgmap v305: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 508 B/s rd, 127 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:07.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:06 smithi144 bash[14758]: cluster 2024-06-30T13:12:06.446361+0000 mgr.y (mgr.14144) 334 : cluster [DBG] pgmap v306: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 102 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:07.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:06 smithi144 bash[19242]: cluster 2024-06-30T13:12:06.446361+0000 mgr.y (mgr.14144) 334 : cluster [DBG] pgmap v306: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 102 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:07.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:06 smithi167 bash[14619]: cluster 2024-06-30T13:12:06.446361+0000 mgr.y (mgr.14144) 334 : cluster [DBG] pgmap v306: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 102 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:09.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:08 smithi144 bash[14758]: cluster 2024-06-30T13:12:08.447454+0000 mgr.y (mgr.14144) 335 : cluster [DBG] pgmap v307: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 833 B/s rd, 92 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:09.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:08 smithi144 bash[19242]: cluster 2024-06-30T13:12:08.447454+0000 mgr.y (mgr.14144) 335 : cluster [DBG] pgmap v307: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 833 B/s rd, 92 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:09.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:08 smithi167 bash[14619]: cluster 2024-06-30T13:12:08.447454+0000 mgr.y (mgr.14144) 335 : cluster [DBG] pgmap v307: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 833 B/s rd, 92 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:11.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:10 smithi144 bash[14758]: cluster 2024-06-30T13:12:10.448805+0000 mgr.y (mgr.14144) 336 : cluster [DBG] pgmap v308: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:11.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:10 smithi144 bash[19242]: cluster 2024-06-30T13:12:10.448805+0000 mgr.y (mgr.14144) 336 : cluster [DBG] pgmap v308: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:11.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:10 smithi167 bash[14619]: cluster 2024-06-30T13:12:10.448805+0000 mgr.y (mgr.14144) 336 : cluster [DBG] pgmap v308: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-06-30T13:12:13.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:12 smithi144 bash[14758]: cluster 2024-06-30T13:12:12.449910+0000 mgr.y (mgr.14144) 337 : cluster [DBG] pgmap v309: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-06-30T13:12:13.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:12 smithi144 bash[19242]: cluster 2024-06-30T13:12:12.449910+0000 mgr.y (mgr.14144) 337 : cluster [DBG] pgmap v309: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-06-30T13:12:13.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:12 smithi167 bash[14619]: cluster 2024-06-30T13:12:12.449910+0000 mgr.y (mgr.14144) 337 : cluster [DBG] pgmap v309: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-06-30T13:12:15.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:14 smithi144 bash[14758]: cluster 2024-06-30T13:12:14.451023+0000 mgr.y (mgr.14144) 338 : cluster [DBG] pgmap v310: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:15.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:14 smithi144 bash[19242]: cluster 2024-06-30T13:12:14.451023+0000 mgr.y (mgr.14144) 338 : cluster [DBG] pgmap v310: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:15.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:14 smithi167 bash[14619]: cluster 2024-06-30T13:12:14.451023+0000 mgr.y (mgr.14144) 338 : cluster [DBG] pgmap v310: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:17.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:16 smithi144 bash[14758]: cluster 2024-06-30T13:12:16.452430+0000 mgr.y (mgr.14144) 339 : cluster [DBG] pgmap v311: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:17.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:16 smithi144 bash[19242]: cluster 2024-06-30T13:12:16.452430+0000 mgr.y (mgr.14144) 339 : cluster [DBG] pgmap v311: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:17.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:16 smithi167 bash[14619]: cluster 2024-06-30T13:12:16.452430+0000 mgr.y (mgr.14144) 339 : cluster [DBG] pgmap v311: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:19.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:18 smithi144 bash[14758]: cluster 2024-06-30T13:12:18.453576+0000 mgr.y (mgr.14144) 340 : cluster [DBG] pgmap v312: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:19.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:18 smithi144 bash[19242]: cluster 2024-06-30T13:12:18.453576+0000 mgr.y (mgr.14144) 340 : cluster [DBG] pgmap v312: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:19.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:18 smithi167 bash[14619]: cluster 2024-06-30T13:12:18.453576+0000 mgr.y (mgr.14144) 340 : cluster [DBG] pgmap v312: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:21.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:20 smithi144 bash[14758]: cluster 2024-06-30T13:12:20.454904+0000 mgr.y (mgr.14144) 341 : cluster [DBG] pgmap v313: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:21.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:20 smithi144 bash[19242]: cluster 2024-06-30T13:12:20.454904+0000 mgr.y (mgr.14144) 341 : cluster [DBG] pgmap v313: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:21.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:20 smithi167 bash[14619]: cluster 2024-06-30T13:12:20.454904+0000 mgr.y (mgr.14144) 341 : cluster [DBG] pgmap v313: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:23.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:22 smithi144 bash[14758]: cluster 2024-06-30T13:12:22.455987+0000 mgr.y (mgr.14144) 342 : cluster [DBG] pgmap v314: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:23.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:22 smithi144 bash[19242]: cluster 2024-06-30T13:12:22.455987+0000 mgr.y (mgr.14144) 342 : cluster [DBG] pgmap v314: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:23.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:22 smithi167 bash[14619]: cluster 2024-06-30T13:12:22.455987+0000 mgr.y (mgr.14144) 342 : cluster [DBG] pgmap v314: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:25.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:24 smithi144 bash[14758]: cluster 2024-06-30T13:12:24.457311+0000 mgr.y (mgr.14144) 343 : cluster [DBG] pgmap v315: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:25.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:24 smithi144 bash[19242]: cluster 2024-06-30T13:12:24.457311+0000 mgr.y (mgr.14144) 343 : cluster [DBG] pgmap v315: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:25.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:24 smithi167 bash[14619]: cluster 2024-06-30T13:12:24.457311+0000 mgr.y (mgr.14144) 343 : cluster [DBG] pgmap v315: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:27.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:26 smithi144 bash[14758]: cluster 2024-06-30T13:12:26.458720+0000 mgr.y (mgr.14144) 344 : cluster [DBG] pgmap v316: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:27.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:26 smithi144 bash[19242]: cluster 2024-06-30T13:12:26.458720+0000 mgr.y (mgr.14144) 344 : cluster [DBG] pgmap v316: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:27.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:26 smithi167 bash[14619]: cluster 2024-06-30T13:12:26.458720+0000 mgr.y (mgr.14144) 344 : cluster [DBG] pgmap v316: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:29.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:28 smithi144 bash[14758]: cluster 2024-06-30T13:12:28.459611+0000 mgr.y (mgr.14144) 345 : cluster [DBG] pgmap v317: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:29.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:28 smithi144 bash[19242]: cluster 2024-06-30T13:12:28.459611+0000 mgr.y (mgr.14144) 345 : cluster [DBG] pgmap v317: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:29.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:28 smithi167 bash[14619]: cluster 2024-06-30T13:12:28.459611+0000 mgr.y (mgr.14144) 345 : cluster [DBG] pgmap v317: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:31.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:30 smithi144 bash[14758]: cluster 2024-06-30T13:12:30.460960+0000 mgr.y (mgr.14144) 346 : cluster [DBG] pgmap v318: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:31.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:30 smithi144 bash[19242]: cluster 2024-06-30T13:12:30.460960+0000 mgr.y (mgr.14144) 346 : cluster [DBG] pgmap v318: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:31.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:30 smithi167 bash[14619]: cluster 2024-06-30T13:12:30.460960+0000 mgr.y (mgr.14144) 346 : cluster [DBG] pgmap v318: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:33.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:32 smithi144 bash[19242]: cluster 2024-06-30T13:12:32.462073+0000 mgr.y (mgr.14144) 347 : cluster [DBG] pgmap v319: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:33.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:32 smithi144 bash[14758]: cluster 2024-06-30T13:12:32.462073+0000 mgr.y (mgr.14144) 347 : cluster [DBG] pgmap v319: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:33.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:32 smithi167 bash[14619]: cluster 2024-06-30T13:12:32.462073+0000 mgr.y (mgr.14144) 347 : cluster [DBG] pgmap v319: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:35.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:34 smithi144 bash[14758]: cluster 2024-06-30T13:12:34.463199+0000 mgr.y (mgr.14144) 348 : cluster [DBG] pgmap v320: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:35.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:34 smithi144 bash[19242]: cluster 2024-06-30T13:12:34.463199+0000 mgr.y (mgr.14144) 348 : cluster [DBG] pgmap v320: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:35.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:34 smithi167 bash[14619]: cluster 2024-06-30T13:12:34.463199+0000 mgr.y (mgr.14144) 348 : cluster [DBG] pgmap v320: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:37.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:36 smithi144 bash[14758]: cluster 2024-06-30T13:12:36.464622+0000 mgr.y (mgr.14144) 349 : cluster [DBG] pgmap v321: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:37.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:36 smithi144 bash[19242]: cluster 2024-06-30T13:12:36.464622+0000 mgr.y (mgr.14144) 349 : cluster [DBG] pgmap v321: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:37.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:36 smithi167 bash[14619]: cluster 2024-06-30T13:12:36.464622+0000 mgr.y (mgr.14144) 349 : cluster [DBG] pgmap v321: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:39.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:38 smithi144 bash[14758]: cluster 2024-06-30T13:12:38.465682+0000 mgr.y (mgr.14144) 350 : cluster [DBG] pgmap v322: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:39.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:38 smithi144 bash[19242]: cluster 2024-06-30T13:12:38.465682+0000 mgr.y (mgr.14144) 350 : cluster [DBG] pgmap v322: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:39.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:38 smithi167 bash[14619]: cluster 2024-06-30T13:12:38.465682+0000 mgr.y (mgr.14144) 350 : cluster [DBG] pgmap v322: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:41.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:40 smithi144 bash[14758]: cluster 2024-06-30T13:12:40.467172+0000 mgr.y (mgr.14144) 351 : cluster [DBG] pgmap v323: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:41.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:40 smithi144 bash[19242]: cluster 2024-06-30T13:12:40.467172+0000 mgr.y (mgr.14144) 351 : cluster [DBG] pgmap v323: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:41.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:40 smithi167 bash[14619]: cluster 2024-06-30T13:12:40.467172+0000 mgr.y (mgr.14144) 351 : cluster [DBG] pgmap v323: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:43.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:42 smithi144 bash[14758]: cluster 2024-06-30T13:12:42.468284+0000 mgr.y (mgr.14144) 352 : cluster [DBG] pgmap v324: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:43.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:42 smithi144 bash[19242]: cluster 2024-06-30T13:12:42.468284+0000 mgr.y (mgr.14144) 352 : cluster [DBG] pgmap v324: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:43.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:42 smithi167 bash[14619]: cluster 2024-06-30T13:12:42.468284+0000 mgr.y (mgr.14144) 352 : cluster [DBG] pgmap v324: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:45.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:44 smithi144 bash[14758]: cluster 2024-06-30T13:12:44.469462+0000 mgr.y (mgr.14144) 353 : cluster [DBG] pgmap v325: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:45.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:44 smithi144 bash[19242]: cluster 2024-06-30T13:12:44.469462+0000 mgr.y (mgr.14144) 353 : cluster [DBG] pgmap v325: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:45.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:44 smithi167 bash[14619]: cluster 2024-06-30T13:12:44.469462+0000 mgr.y (mgr.14144) 353 : cluster [DBG] pgmap v325: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:47.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:46 smithi144 bash[14758]: cluster 2024-06-30T13:12:46.470891+0000 mgr.y (mgr.14144) 354 : cluster [DBG] pgmap v326: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:47.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:46 smithi144 bash[19242]: cluster 2024-06-30T13:12:46.470891+0000 mgr.y (mgr.14144) 354 : cluster [DBG] pgmap v326: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:47.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:46 smithi167 bash[14619]: cluster 2024-06-30T13:12:46.470891+0000 mgr.y (mgr.14144) 354 : cluster [DBG] pgmap v326: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:49.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:48 smithi144 bash[19242]: cluster 2024-06-30T13:12:48.471813+0000 mgr.y (mgr.14144) 355 : cluster [DBG] pgmap v327: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:49.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:48 smithi144 bash[14758]: cluster 2024-06-30T13:12:48.471813+0000 mgr.y (mgr.14144) 355 : cluster [DBG] pgmap v327: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:49.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:48 smithi167 bash[14619]: cluster 2024-06-30T13:12:48.471813+0000 mgr.y (mgr.14144) 355 : cluster [DBG] pgmap v327: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:51.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:50 smithi144 bash[14758]: cluster 2024-06-30T13:12:50.473098+0000 mgr.y (mgr.14144) 356 : cluster [DBG] pgmap v328: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:51.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:50 smithi144 bash[19242]: cluster 2024-06-30T13:12:50.473098+0000 mgr.y (mgr.14144) 356 : cluster [DBG] pgmap v328: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:51.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:50 smithi167 bash[14619]: cluster 2024-06-30T13:12:50.473098+0000 mgr.y (mgr.14144) 356 : cluster [DBG] pgmap v328: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:53.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:52 smithi144 bash[14758]: cluster 2024-06-30T13:12:52.474181+0000 mgr.y (mgr.14144) 357 : cluster [DBG] pgmap v329: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:53.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:52 smithi144 bash[19242]: cluster 2024-06-30T13:12:52.474181+0000 mgr.y (mgr.14144) 357 : cluster [DBG] pgmap v329: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:53.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:52 smithi167 bash[14619]: cluster 2024-06-30T13:12:52.474181+0000 mgr.y (mgr.14144) 357 : cluster [DBG] pgmap v329: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:55.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:54 smithi144 bash[14758]: cluster 2024-06-30T13:12:54.475515+0000 mgr.y (mgr.14144) 358 : cluster [DBG] pgmap v330: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:55.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:54 smithi144 bash[19242]: cluster 2024-06-30T13:12:54.475515+0000 mgr.y (mgr.14144) 358 : cluster [DBG] pgmap v330: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:55.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:54 smithi167 bash[14619]: cluster 2024-06-30T13:12:54.475515+0000 mgr.y (mgr.14144) 358 : cluster [DBG] pgmap v330: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:56 smithi144 bash[14758]: cluster 2024-06-30T13:12:56.476674+0000 mgr.y (mgr.14144) 359 : cluster [DBG] pgmap v331: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:56 smithi144 bash[14758]: audit 2024-06-30T13:12:56.537497+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:12:57.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:56 smithi144 bash[14758]: audit 2024-06-30T13:12:56.538097+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:12:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:56 smithi144 bash[19242]: cluster 2024-06-30T13:12:56.476674+0000 mgr.y (mgr.14144) 359 : cluster [DBG] pgmap v331: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:56 smithi144 bash[19242]: audit 2024-06-30T13:12:56.537497+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:12:57.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:56 smithi144 bash[19242]: audit 2024-06-30T13:12:56.538097+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:12:57.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:56 smithi167 bash[14619]: cluster 2024-06-30T13:12:56.476674+0000 mgr.y (mgr.14144) 359 : cluster [DBG] pgmap v331: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:12:57.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:56 smithi167 bash[14619]: audit 2024-06-30T13:12:56.537497+0000 mon.a (mon.0) 616 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:12:57.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:56 smithi167 bash[14619]: audit 2024-06-30T13:12:56.538097+0000 mon.a (mon.0) 617 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:12:59.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:12:58 smithi144 bash[14758]: cluster 2024-06-30T13:12:58.477702+0000 mgr.y (mgr.14144) 360 : cluster [DBG] pgmap v332: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:59.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:12:58 smithi144 bash[19242]: cluster 2024-06-30T13:12:58.477702+0000 mgr.y (mgr.14144) 360 : cluster [DBG] pgmap v332: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:12:59.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:12:58 smithi167 bash[14619]: cluster 2024-06-30T13:12:58.477702+0000 mgr.y (mgr.14144) 360 : cluster [DBG] pgmap v332: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:01.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:00 smithi144 bash[14758]: audit 2024-06-30T13:13:00.163422+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:13:01.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:00 smithi144 bash[14758]: audit 2024-06-30T13:13:00.443202+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:13:01.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:00 smithi144 bash[14758]: cluster 2024-06-30T13:13:00.479026+0000 mgr.y (mgr.14144) 361 : cluster [DBG] pgmap v333: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:01.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:00 smithi144 bash[19242]: audit 2024-06-30T13:13:00.163422+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:13:01.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:00 smithi144 bash[19242]: audit 2024-06-30T13:13:00.443202+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:13:01.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:00 smithi144 bash[19242]: cluster 2024-06-30T13:13:00.479026+0000 mgr.y (mgr.14144) 361 : cluster [DBG] pgmap v333: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:01.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:00 smithi167 bash[14619]: audit 2024-06-30T13:13:00.163422+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:13:01.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:00 smithi167 bash[14619]: audit 2024-06-30T13:13:00.443202+0000 mon.a (mon.0) 619 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:13:01.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:00 smithi167 bash[14619]: cluster 2024-06-30T13:13:00.479026+0000 mgr.y (mgr.14144) 361 : cluster [DBG] pgmap v333: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:03.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:02 smithi144 bash[14758]: cluster 2024-06-30T13:13:02.480097+0000 mgr.y (mgr.14144) 362 : cluster [DBG] pgmap v334: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:03.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:02 smithi144 bash[19242]: cluster 2024-06-30T13:13:02.480097+0000 mgr.y (mgr.14144) 362 : cluster [DBG] pgmap v334: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:03.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:02 smithi167 bash[14619]: cluster 2024-06-30T13:13:02.480097+0000 mgr.y (mgr.14144) 362 : cluster [DBG] pgmap v334: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:05.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:04 smithi144 bash[14758]: cluster 2024-06-30T13:13:04.481427+0000 mgr.y (mgr.14144) 363 : cluster [DBG] pgmap v335: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:05.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:04 smithi144 bash[19242]: cluster 2024-06-30T13:13:04.481427+0000 mgr.y (mgr.14144) 363 : cluster [DBG] pgmap v335: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:05.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:04 smithi167 bash[14619]: cluster 2024-06-30T13:13:04.481427+0000 mgr.y (mgr.14144) 363 : cluster [DBG] pgmap v335: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:07.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:06 smithi144 bash[14758]: cluster 2024-06-30T13:13:06.482632+0000 mgr.y (mgr.14144) 364 : cluster [DBG] pgmap v336: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:07.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:06 smithi144 bash[19242]: cluster 2024-06-30T13:13:06.482632+0000 mgr.y (mgr.14144) 364 : cluster [DBG] pgmap v336: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:07.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:06 smithi167 bash[14619]: cluster 2024-06-30T13:13:06.482632+0000 mgr.y (mgr.14144) 364 : cluster [DBG] pgmap v336: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:09.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:08 smithi144 bash[14758]: cluster 2024-06-30T13:13:08.483540+0000 mgr.y (mgr.14144) 365 : cluster [DBG] pgmap v337: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:09.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:08 smithi144 bash[19242]: cluster 2024-06-30T13:13:08.483540+0000 mgr.y (mgr.14144) 365 : cluster [DBG] pgmap v337: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:09.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:08 smithi167 bash[14619]: cluster 2024-06-30T13:13:08.483540+0000 mgr.y (mgr.14144) 365 : cluster [DBG] pgmap v337: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:11.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:10 smithi144 bash[14758]: cluster 2024-06-30T13:13:10.484886+0000 mgr.y (mgr.14144) 366 : cluster [DBG] pgmap v338: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:11.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:10 smithi144 bash[19242]: cluster 2024-06-30T13:13:10.484886+0000 mgr.y (mgr.14144) 366 : cluster [DBG] pgmap v338: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:11.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:10 smithi167 bash[14619]: cluster 2024-06-30T13:13:10.484886+0000 mgr.y (mgr.14144) 366 : cluster [DBG] pgmap v338: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:13.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:12 smithi144 bash[19242]: cluster 2024-06-30T13:13:12.486013+0000 mgr.y (mgr.14144) 367 : cluster [DBG] pgmap v339: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:13.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:12 smithi144 bash[14758]: cluster 2024-06-30T13:13:12.486013+0000 mgr.y (mgr.14144) 367 : cluster [DBG] pgmap v339: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:13.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:12 smithi167 bash[14619]: cluster 2024-06-30T13:13:12.486013+0000 mgr.y (mgr.14144) 367 : cluster [DBG] pgmap v339: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:15.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:14 smithi144 bash[14758]: cluster 2024-06-30T13:13:14.487329+0000 mgr.y (mgr.14144) 368 : cluster [DBG] pgmap v340: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:15.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:14 smithi144 bash[19242]: cluster 2024-06-30T13:13:14.487329+0000 mgr.y (mgr.14144) 368 : cluster [DBG] pgmap v340: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:15.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:14 smithi167 bash[14619]: cluster 2024-06-30T13:13:14.487329+0000 mgr.y (mgr.14144) 368 : cluster [DBG] pgmap v340: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:17.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:16 smithi144 bash[14758]: cluster 2024-06-30T13:13:16.488601+0000 mgr.y (mgr.14144) 369 : cluster [DBG] pgmap v341: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:17.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:16 smithi144 bash[19242]: cluster 2024-06-30T13:13:16.488601+0000 mgr.y (mgr.14144) 369 : cluster [DBG] pgmap v341: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:17.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:16 smithi167 bash[14619]: cluster 2024-06-30T13:13:16.488601+0000 mgr.y (mgr.14144) 369 : cluster [DBG] pgmap v341: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:19.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:18 smithi144 bash[14758]: cluster 2024-06-30T13:13:18.489808+0000 mgr.y (mgr.14144) 370 : cluster [DBG] pgmap v342: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:19.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:18 smithi144 bash[19242]: cluster 2024-06-30T13:13:18.489808+0000 mgr.y (mgr.14144) 370 : cluster [DBG] pgmap v342: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:19.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:18 smithi167 bash[14619]: cluster 2024-06-30T13:13:18.489808+0000 mgr.y (mgr.14144) 370 : cluster [DBG] pgmap v342: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:21.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:20 smithi144 bash[14758]: cluster 2024-06-30T13:13:20.491177+0000 mgr.y (mgr.14144) 371 : cluster [DBG] pgmap v343: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:21.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:20 smithi144 bash[19242]: cluster 2024-06-30T13:13:20.491177+0000 mgr.y (mgr.14144) 371 : cluster [DBG] pgmap v343: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:21.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:20 smithi167 bash[14619]: cluster 2024-06-30T13:13:20.491177+0000 mgr.y (mgr.14144) 371 : cluster [DBG] pgmap v343: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:23.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:22 smithi144 bash[14758]: cluster 2024-06-30T13:13:22.492262+0000 mgr.y (mgr.14144) 372 : cluster [DBG] pgmap v344: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:23.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:22 smithi144 bash[19242]: cluster 2024-06-30T13:13:22.492262+0000 mgr.y (mgr.14144) 372 : cluster [DBG] pgmap v344: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:23.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:22 smithi167 bash[14619]: cluster 2024-06-30T13:13:22.492262+0000 mgr.y (mgr.14144) 372 : cluster [DBG] pgmap v344: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:25.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:24 smithi144 bash[19242]: cluster 2024-06-30T13:13:24.493600+0000 mgr.y (mgr.14144) 373 : cluster [DBG] pgmap v345: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:25.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:24 smithi144 bash[14758]: cluster 2024-06-30T13:13:24.493600+0000 mgr.y (mgr.14144) 373 : cluster [DBG] pgmap v345: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:25.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:24 smithi167 bash[14619]: cluster 2024-06-30T13:13:24.493600+0000 mgr.y (mgr.14144) 373 : cluster [DBG] pgmap v345: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:27.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:26 smithi144 bash[14758]: cluster 2024-06-30T13:13:26.494805+0000 mgr.y (mgr.14144) 374 : cluster [DBG] pgmap v346: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:27.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:26 smithi144 bash[19242]: cluster 2024-06-30T13:13:26.494805+0000 mgr.y (mgr.14144) 374 : cluster [DBG] pgmap v346: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:27.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:26 smithi167 bash[14619]: cluster 2024-06-30T13:13:26.494805+0000 mgr.y (mgr.14144) 374 : cluster [DBG] pgmap v346: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:29.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:28 smithi144 bash[14758]: cluster 2024-06-30T13:13:28.495845+0000 mgr.y (mgr.14144) 375 : cluster [DBG] pgmap v347: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:29.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:28 smithi144 bash[19242]: cluster 2024-06-30T13:13:28.495845+0000 mgr.y (mgr.14144) 375 : cluster [DBG] pgmap v347: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:29.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:28 smithi167 bash[14619]: cluster 2024-06-30T13:13:28.495845+0000 mgr.y (mgr.14144) 375 : cluster [DBG] pgmap v347: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:31.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:30 smithi144 bash[14758]: cluster 2024-06-30T13:13:30.497024+0000 mgr.y (mgr.14144) 376 : cluster [DBG] pgmap v348: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:31.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:30 smithi144 bash[19242]: cluster 2024-06-30T13:13:30.497024+0000 mgr.y (mgr.14144) 376 : cluster [DBG] pgmap v348: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:31.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:30 smithi167 bash[14619]: cluster 2024-06-30T13:13:30.497024+0000 mgr.y (mgr.14144) 376 : cluster [DBG] pgmap v348: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:33.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:32 smithi144 bash[19242]: cluster 2024-06-30T13:13:32.498324+0000 mgr.y (mgr.14144) 377 : cluster [DBG] pgmap v349: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:33.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:32 smithi144 bash[14758]: cluster 2024-06-30T13:13:32.498324+0000 mgr.y (mgr.14144) 377 : cluster [DBG] pgmap v349: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:33.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:32 smithi167 bash[14619]: cluster 2024-06-30T13:13:32.498324+0000 mgr.y (mgr.14144) 377 : cluster [DBG] pgmap v349: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:35.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:34 smithi144 bash[19242]: cluster 2024-06-30T13:13:34.499717+0000 mgr.y (mgr.14144) 378 : cluster [DBG] pgmap v350: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:35.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:34 smithi144 bash[14758]: cluster 2024-06-30T13:13:34.499717+0000 mgr.y (mgr.14144) 378 : cluster [DBG] pgmap v350: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:35.027 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:34 smithi167 bash[14619]: cluster 2024-06-30T13:13:34.499717+0000 mgr.y (mgr.14144) 378 : cluster [DBG] pgmap v350: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:37.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:36 smithi144 bash[19242]: cluster 2024-06-30T13:13:36.500973+0000 mgr.y (mgr.14144) 379 : cluster [DBG] pgmap v351: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:37.011 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:36 smithi144 bash[14758]: cluster 2024-06-30T13:13:36.500973+0000 mgr.y (mgr.14144) 379 : cluster [DBG] pgmap v351: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:37.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:36 smithi167 bash[14619]: cluster 2024-06-30T13:13:36.500973+0000 mgr.y (mgr.14144) 379 : cluster [DBG] pgmap v351: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:40.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:39 smithi144 bash[19242]: cluster 2024-06-30T13:13:38.502150+0000 mgr.y (mgr.14144) 380 : cluster [DBG] pgmap v352: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:40.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:39 smithi144 bash[14758]: cluster 2024-06-30T13:13:38.502150+0000 mgr.y (mgr.14144) 380 : cluster [DBG] pgmap v352: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:40.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:39 smithi167 bash[14619]: cluster 2024-06-30T13:13:38.502150+0000 mgr.y (mgr.14144) 380 : cluster [DBG] pgmap v352: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:42.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:41 smithi144 bash[14758]: cluster 2024-06-30T13:13:40.503588+0000 mgr.y (mgr.14144) 381 : cluster [DBG] pgmap v353: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:42.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:41 smithi144 bash[19242]: cluster 2024-06-30T13:13:40.503588+0000 mgr.y (mgr.14144) 381 : cluster [DBG] pgmap v353: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:42.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:41 smithi167 bash[14619]: cluster 2024-06-30T13:13:40.503588+0000 mgr.y (mgr.14144) 381 : cluster [DBG] pgmap v353: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:44.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:43 smithi144 bash[14758]: cluster 2024-06-30T13:13:42.504772+0000 mgr.y (mgr.14144) 382 : cluster [DBG] pgmap v354: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:44.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:43 smithi144 bash[19242]: cluster 2024-06-30T13:13:42.504772+0000 mgr.y (mgr.14144) 382 : cluster [DBG] pgmap v354: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:44.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:43 smithi167 bash[14619]: cluster 2024-06-30T13:13:42.504772+0000 mgr.y (mgr.14144) 382 : cluster [DBG] pgmap v354: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:46.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:45 smithi144 bash[14758]: cluster 2024-06-30T13:13:44.506048+0000 mgr.y (mgr.14144) 383 : cluster [DBG] pgmap v355: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:46.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:45 smithi144 bash[19242]: cluster 2024-06-30T13:13:44.506048+0000 mgr.y (mgr.14144) 383 : cluster [DBG] pgmap v355: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:46.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:45 smithi167 bash[14619]: cluster 2024-06-30T13:13:44.506048+0000 mgr.y (mgr.14144) 383 : cluster [DBG] pgmap v355: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:48.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:47 smithi144 bash[14758]: cluster 2024-06-30T13:13:46.507239+0000 mgr.y (mgr.14144) 384 : cluster [DBG] pgmap v356: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:48.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:47 smithi144 bash[19242]: cluster 2024-06-30T13:13:46.507239+0000 mgr.y (mgr.14144) 384 : cluster [DBG] pgmap v356: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:48.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:47 smithi167 bash[14619]: cluster 2024-06-30T13:13:46.507239+0000 mgr.y (mgr.14144) 384 : cluster [DBG] pgmap v356: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:50.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:49 smithi167 bash[14619]: cluster 2024-06-30T13:13:48.508351+0000 mgr.y (mgr.14144) 385 : cluster [DBG] pgmap v357: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:50.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:49 smithi144 bash[14758]: cluster 2024-06-30T13:13:48.508351+0000 mgr.y (mgr.14144) 385 : cluster [DBG] pgmap v357: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:50.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:49 smithi144 bash[19242]: cluster 2024-06-30T13:13:48.508351+0000 mgr.y (mgr.14144) 385 : cluster [DBG] pgmap v357: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:52.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:51 smithi167 bash[14619]: cluster 2024-06-30T13:13:50.509654+0000 mgr.y (mgr.14144) 386 : cluster [DBG] pgmap v358: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:52.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:51 smithi144 bash[19242]: cluster 2024-06-30T13:13:50.509654+0000 mgr.y (mgr.14144) 386 : cluster [DBG] pgmap v358: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:52.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:51 smithi144 bash[14758]: cluster 2024-06-30T13:13:50.509654+0000 mgr.y (mgr.14144) 386 : cluster [DBG] pgmap v358: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:54.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:53 smithi167 bash[14619]: cluster 2024-06-30T13:13:52.510715+0000 mgr.y (mgr.14144) 387 : cluster [DBG] pgmap v359: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:54.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:53 smithi144 bash[19242]: cluster 2024-06-30T13:13:52.510715+0000 mgr.y (mgr.14144) 387 : cluster [DBG] pgmap v359: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:54.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:53 smithi144 bash[14758]: cluster 2024-06-30T13:13:52.510715+0000 mgr.y (mgr.14144) 387 : cluster [DBG] pgmap v359: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:56.219 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:55 smithi144 bash[19242]: cluster 2024-06-30T13:13:54.511986+0000 mgr.y (mgr.14144) 388 : cluster [DBG] pgmap v360: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:56.219 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:55 smithi144 bash[14758]: cluster 2024-06-30T13:13:54.511986+0000 mgr.y (mgr.14144) 388 : cluster [DBG] pgmap v360: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:56.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:55 smithi167 bash[14619]: cluster 2024-06-30T13:13:54.511986+0000 mgr.y (mgr.14144) 388 : cluster [DBG] pgmap v360: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:13:56.711 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-06-30T13:13:56.973 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:56 smithi144 bash[14758]: audit 2024-06-30T13:13:56.547884+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:13:56.973 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:56 smithi144 bash[14758]: audit 2024-06-30T13:13:56.548217+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:13:56.973 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:56 smithi144 bash[19242]: audit 2024-06-30T13:13:56.547884+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:13:56.973 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:56 smithi144 bash[19242]: audit 2024-06-30T13:13:56.548217+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:13:57.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:56 smithi167 bash[14619]: audit 2024-06-30T13:13:56.547884+0000 mon.a (mon.0) 620 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:13:57.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:56 smithi167 bash[14619]: audit 2024-06-30T13:13:56.548217+0000 mon.a (mon.0) 621 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:13:58.048 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:57 smithi144 bash[14758]: cluster 2024-06-30T13:13:56.513233+0000 mgr.y (mgr.14144) 389 : cluster [DBG] pgmap v361: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:58.048 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:57 smithi144 bash[19242]: cluster 2024-06-30T13:13:56.513233+0000 mgr.y (mgr.14144) 389 : cluster [DBG] pgmap v361: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:13:58.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:57 smithi167 bash[14619]: cluster 2024-06-30T13:13:56.513233+0000 mgr.y (mgr.14144) 389 : cluster [DBG] pgmap v361: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:00.124 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:13:59 smithi144 bash[14758]: cluster 2024-06-30T13:13:58.514341+0000 mgr.y (mgr.14144) 390 : cluster [DBG] pgmap v362: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:00.124 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:13:59 smithi144 bash[19242]: cluster 2024-06-30T13:13:58.514341+0000 mgr.y (mgr.14144) 390 : cluster [DBG] pgmap v362: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:00.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:13:59 smithi167 bash[14619]: cluster 2024-06-30T13:13:58.514341+0000 mgr.y (mgr.14144) 390 : cluster [DBG] pgmap v362: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:00.802 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:00 smithi144 bash[14758]: audit 2024-06-30T13:14:00.451469+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:00.803 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:00 smithi144 bash[14758]: audit 2024-06-30T13:14:00.519368+0000 mon.a (mon.0) 623 : audit [INF] from='client.? ' entity='client.admin' 2024-06-30T13:14:00.803 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:00 smithi144 bash[14758]: audit 2024-06-30T13:14:00.702774+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:01.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:00 smithi144 bash[19242]: audit 2024-06-30T13:14:00.451469+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:01.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:00 smithi144 bash[19242]: audit 2024-06-30T13:14:00.519368+0000 mon.a (mon.0) 623 : audit [INF] from='client.? ' entity='client.admin' 2024-06-30T13:14:01.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:00 smithi144 bash[19242]: audit 2024-06-30T13:14:00.702774+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:01.245 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-06-30T13:14:01.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:00 smithi167 bash[14619]: audit 2024-06-30T13:14:00.451469+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:01.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:00 smithi167 bash[14619]: audit 2024-06-30T13:14:00.519368+0000 mon.a (mon.0) 623 : audit [INF] from='client.? ' entity='client.admin' 2024-06-30T13:14:01.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:00 smithi167 bash[14619]: audit 2024-06-30T13:14:00.702774+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:02.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:01 smithi144 bash[19242]: cluster 2024-06-30T13:14:00.515174+0000 mgr.y (mgr.14144) 391 : cluster [DBG] pgmap v363: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:02.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:01 smithi144 bash[19242]: audit 2024-06-30T13:14:00.935273+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:02.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:01 smithi144 bash[14758]: cluster 2024-06-30T13:14:00.515174+0000 mgr.y (mgr.14144) 391 : cluster [DBG] pgmap v363: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:02.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:01 smithi144 bash[14758]: audit 2024-06-30T13:14:00.935273+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:02.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:01 smithi167 bash[14619]: cluster 2024-06-30T13:14:00.515174+0000 mgr.y (mgr.14144) 391 : cluster [DBG] pgmap v363: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:02.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:01 smithi167 bash[14619]: audit 2024-06-30T13:14:00.935273+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:04.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:03 smithi144 bash[14758]: cluster 2024-06-30T13:14:02.516157+0000 mgr.y (mgr.14144) 392 : cluster [DBG] pgmap v364: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:04.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:03 smithi144 bash[19242]: cluster 2024-06-30T13:14:02.516157+0000 mgr.y (mgr.14144) 392 : cluster [DBG] pgmap v364: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:04.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:03 smithi167 bash[14619]: cluster 2024-06-30T13:14:02.516157+0000 mgr.y (mgr.14144) 392 : cluster [DBG] pgmap v364: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:05.841 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph config set global log_to_journald false --force' 2024-06-30T13:14:06.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:06 smithi144 bash[14758]: cluster 2024-06-30T13:14:04.517406+0000 mgr.y (mgr.14144) 393 : cluster [DBG] pgmap v365: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:06.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:06 smithi144 bash[14758]: audit 2024-06-30T13:14:05.117087+0000 mon.a (mon.0) 626 : audit [INF] from='client.? ' entity='client.admin' 2024-06-30T13:14:06.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:06 smithi144 bash[19242]: cluster 2024-06-30T13:14:04.517406+0000 mgr.y (mgr.14144) 393 : cluster [DBG] pgmap v365: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:06.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:06 smithi144 bash[19242]: audit 2024-06-30T13:14:05.117087+0000 mon.a (mon.0) 626 : audit [INF] from='client.? ' entity='client.admin' 2024-06-30T13:14:06.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:06 smithi167 bash[14619]: cluster 2024-06-30T13:14:04.517406+0000 mgr.y (mgr.14144) 393 : cluster [DBG] pgmap v365: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:06.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:06 smithi167 bash[14619]: audit 2024-06-30T13:14:05.117087+0000 mon.a (mon.0) 626 : audit [INF] from='client.? ' entity='client.admin' 2024-06-30T13:14:08.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:08 smithi144 bash[19242]: cluster 2024-06-30T13:14:06.518590+0000 mgr.y (mgr.14144) 394 : cluster [DBG] pgmap v366: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:08.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:08 smithi144 bash[14758]: cluster 2024-06-30T13:14:06.518590+0000 mgr.y (mgr.14144) 394 : cluster [DBG] pgmap v366: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:08.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:08 smithi167 bash[14619]: cluster 2024-06-30T13:14:06.518590+0000 mgr.y (mgr.14144) 394 : cluster [DBG] pgmap v366: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:10.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:10 smithi144 bash[14758]: cluster 2024-06-30T13:14:08.519707+0000 mgr.y (mgr.14144) 395 : cluster [DBG] pgmap v367: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:10.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:10 smithi144 bash[14758]: audit 2024-06-30T13:14:09.676665+0000 mon.a (mon.0) 627 : audit [INF] from='client.? ' entity='client.admin' 2024-06-30T13:14:10.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:10 smithi144 bash[14758]: audit 2024-06-30T13:14:09.682468+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:10.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:10 smithi144 bash[14758]: audit 2024-06-30T13:14:09.691483+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:10.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:10 smithi144 bash[19242]: cluster 2024-06-30T13:14:08.519707+0000 mgr.y (mgr.14144) 395 : cluster [DBG] pgmap v367: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:10.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:10 smithi144 bash[19242]: audit 2024-06-30T13:14:09.676665+0000 mon.a (mon.0) 627 : audit [INF] from='client.? ' entity='client.admin' 2024-06-30T13:14:10.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:10 smithi144 bash[19242]: audit 2024-06-30T13:14:09.682468+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:10.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:10 smithi144 bash[19242]: audit 2024-06-30T13:14:09.691483+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:10.405 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-06-30T13:14:10.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:10 smithi167 bash[14619]: cluster 2024-06-30T13:14:08.519707+0000 mgr.y (mgr.14144) 395 : cluster [DBG] pgmap v367: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:10.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:10 smithi167 bash[14619]: audit 2024-06-30T13:14:09.676665+0000 mon.a (mon.0) 627 : audit [INF] from='client.? ' entity='client.admin' 2024-06-30T13:14:10.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:10 smithi167 bash[14619]: audit 2024-06-30T13:14:09.682468+0000 mon.a (mon.0) 628 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:10.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:10 smithi167 bash[14619]: audit 2024-06-30T13:14:09.691483+0000 mon.a (mon.0) 629 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:12.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:12 smithi144 bash[14758]: cluster 2024-06-30T13:14:10.520948+0000 mgr.y (mgr.14144) 396 : cluster [DBG] pgmap v368: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:12.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:12 smithi144 bash[19242]: cluster 2024-06-30T13:14:10.520948+0000 mgr.y (mgr.14144) 396 : cluster [DBG] pgmap v368: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:12.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:12 smithi167 bash[14619]: cluster 2024-06-30T13:14:10.520948+0000 mgr.y (mgr.14144) 396 : cluster [DBG] pgmap v368: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:14.296 INFO:teuthology.orchestra.run.smithi144.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:14.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:14 smithi144 bash[14758]: cluster 2024-06-30T13:14:12.522102+0000 mgr.y (mgr.14144) 397 : cluster [DBG] pgmap v369: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:14.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:14 smithi144 bash[19242]: cluster 2024-06-30T13:14:12.522102+0000 mgr.y (mgr.14144) 397 : cluster [DBG] pgmap v369: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:14.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:14 smithi167 bash[14619]: cluster 2024-06-30T13:14:12.522102+0000 mgr.y (mgr.14144) 397 : cluster [DBG] pgmap v369: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:15.164 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-30T13:14:15.174 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi144.front.sepia.ceph.com 2024-06-30T13:14:15.174 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-06-30T13:14:15.261 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: debug 2024-06-30T13:14:15.095+0000 7f017b284700 -1 log_channel(cephadm) log [ERR] : cephadm exited with an error code: 1, stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65... 2024-06-30T13:14:15.261 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:15.261 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: /usr/bin/docker: stderr Error response from daemon: unknown: Tag f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 was deleted or has expired. To pull, revive via time machine 2024-06-30T13:14:15.261 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: Traceback (most recent call last): 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7924, in 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: main() 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7912, in main 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: r = ctx.func(ctx) 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 1695, in _infer_image 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: return func(ctx) 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3196, in command_pull 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: _pull_image(ctx, ctx.image) 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3221, in _pull_image 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: raise RuntimeError('Failed command: %s' % cmd_str) 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: RuntimeError: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: Traceback (most recent call last): 2024-06-30T13:14:15.262 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1172, in _remote_connection 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: yield (conn, connr) 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1087, in _run_cephadm 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: code, '\n'.join(err))) 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65... 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: /usr/bin/docker: stderr Error response from daemon: unknown: Tag f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 was deleted or has expired. To pull, revive via time machine 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: Traceback (most recent call last): 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7924, in 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: main() 2024-06-30T13:14:15.263 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7912, in main 2024-06-30T13:14:15.264 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: r = ctx.func(ctx) 2024-06-30T13:14:15.264 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 1695, in _infer_image 2024-06-30T13:14:15.264 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: return func(ctx) 2024-06-30T13:14:15.264 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3196, in command_pull 2024-06-30T13:14:15.264 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: _pull_image(ctx, ctx.image) 2024-06-30T13:14:15.264 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3221, in _pull_image 2024-06-30T13:14:15.264 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: raise RuntimeError('Failed command: %s' % cmd_str) 2024-06-30T13:14:15.264 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: RuntimeError: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:15.264 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[15024]: debug 2024-06-30T13:14:15.095+0000 7f017b284700 -1 log_channel(cephadm) log [ERR] : Upgrade: Paused due to UPGRADE_FAILED_PULL: Upgrade: failed to pull target image 2024-06-30T13:14:15.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[19242]: audit 2024-06-30T13:14:14.283222+0000 mgr.y (mgr.14144) 398 : audit [DBG] from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[19242]: cephadm 2024-06-30T13:14:14.284567+0000 mgr.y (mgr.14144) 399 : cephadm [INF] Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[19242]: audit 2024-06-30T13:14:14.292495+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[19242]: audit 2024-06-30T13:14:14.293661+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[19242]: audit 2024-06-30T13:14:14.551302+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[19242]: audit 2024-06-30T13:14:14.559102+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[19242]: audit 2024-06-30T13:14:15.103845+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[19242]: audit 2024-06-30T13:14:15.107151+0000 mon.a (mon.0) 635 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[19242]: audit 2024-06-30T13:14:15.116502+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[14758]: audit 2024-06-30T13:14:14.283222+0000 mgr.y (mgr.14144) 398 : audit [DBG] from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[14758]: cephadm 2024-06-30T13:14:14.284567+0000 mgr.y (mgr.14144) 399 : cephadm [INF] Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:15.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[14758]: audit 2024-06-30T13:14:14.292495+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[14758]: audit 2024-06-30T13:14:14.293661+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:15.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[14758]: audit 2024-06-30T13:14:14.551302+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[14758]: audit 2024-06-30T13:14:14.559102+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[14758]: audit 2024-06-30T13:14:15.103845+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[14758]: audit 2024-06-30T13:14:15.107151+0000 mon.a (mon.0) 635 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:15.762 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:15 smithi144 bash[14758]: audit 2024-06-30T13:14:15.116502+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:15 smithi167 bash[14619]: audit 2024-06-30T13:14:14.283222+0000 mgr.y (mgr.14144) 398 : audit [DBG] from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:15 smithi167 bash[14619]: cephadm 2024-06-30T13:14:14.284567+0000 mgr.y (mgr.14144) 399 : cephadm [INF] Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:15 smithi167 bash[14619]: audit 2024-06-30T13:14:14.292495+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:15 smithi167 bash[14619]: audit 2024-06-30T13:14:14.293661+0000 mon.a (mon.0) 631 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:15 smithi167 bash[14619]: audit 2024-06-30T13:14:14.551302+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:15 smithi167 bash[14619]: audit 2024-06-30T13:14:14.559102+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:15 smithi167 bash[14619]: audit 2024-06-30T13:14:15.103845+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:15 smithi167 bash[14619]: audit 2024-06-30T13:14:15.107151+0000 mon.a (mon.0) 635 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:14:15.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:15 smithi167 bash[14619]: audit 2024-06-30T13:14:15.116502+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: cluster 2024-06-30T13:14:14.523327+0000 mgr.y (mgr.14144) 400 : cluster [DBG] pgmap v370: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: cephadm 2024-06-30T13:14:14.565358+0000 mgr.y (mgr.14144) 401 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: cephadm 2024-06-30T13:14:15.098284+0000 mgr.y (mgr.14144) 402 : cephadm [ERR] cephadm exited with an error code: 1, stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65... 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: /usr/bin/docker: stderr Error response from daemon: unknown: Tag f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 was deleted or has expired. To pull, revive via time machine 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: Traceback (most recent call last): 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7924, in 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: main() 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7912, in main 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: r = ctx.func(ctx) 2024-06-30T13:14:16.584 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 1695, in _infer_image 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: return func(ctx) 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3196, in command_pull 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: _pull_image(ctx, ctx.image) 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3221, in _pull_image 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: raise RuntimeError('Failed command: %s' % cmd_str) 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: RuntimeError: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: Traceback (most recent call last): 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1172, in _remote_connection 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: yield (conn, connr) 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1087, in _run_cephadm 2024-06-30T13:14:16.585 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: code, '\n'.join(err))) 2024-06-30T13:14:16.586 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65... 2024-06-30T13:14:16.586 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.586 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: /usr/bin/docker: stderr Error response from daemon: unknown: Tag f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 was deleted or has expired. To pull, revive via time machine 2024-06-30T13:14:16.586 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: Traceback (most recent call last): 2024-06-30T13:14:16.586 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7924, in 2024-06-30T13:14:16.586 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: main() 2024-06-30T13:14:16.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: cluster 2024-06-30T13:14:14.523327+0000 mgr.y (mgr.14144) 400 : cluster [DBG] pgmap v370: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:16.586 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: cephadm 2024-06-30T13:14:14.565358+0000 mgr.y (mgr.14144) 401 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: cephadm 2024-06-30T13:14:15.098284+0000 mgr.y (mgr.14144) 402 : cephadm [ERR] cephadm exited with an error code: 1, stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65... 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: /usr/bin/docker: stderr Error response from daemon: unknown: Tag f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 was deleted or has expired. To pull, revive via time machine 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: Traceback (most recent call last): 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7924, in 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: main() 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7912, in main 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: r = ctx.func(ctx) 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 1695, in _infer_image 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: return func(ctx) 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3196, in command_pull 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: _pull_image(ctx, ctx.image) 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3221, in _pull_image 2024-06-30T13:14:16.587 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: raise RuntimeError('Failed command: %s' % cmd_str) 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: RuntimeError: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: Traceback (most recent call last): 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1172, in _remote_connection 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: yield (conn, connr) 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1087, in _run_cephadm 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: code, '\n'.join(err))) 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65... 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: /usr/bin/docker: stderr Error response from daemon: unknown: Tag f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 was deleted or has expired. To pull, revive via time machine 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: Traceback (most recent call last): 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7924, in 2024-06-30T13:14:16.588 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: main() 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7912, in main 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: r = ctx.func(ctx) 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 1695, in _infer_image 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: return func(ctx) 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3196, in command_pull 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: _pull_image(ctx, ctx.image) 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3221, in _pull_image 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: raise RuntimeError('Failed command: %s' % cmd_str) 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: RuntimeError: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: cephadm 2024-06-30T13:14:15.098568+0000 mgr.y (mgr.14144) 403 : cephadm [ERR] Upgrade: Paused due to UPGRADE_FAILED_PULL: Upgrade: failed to pull target image 2024-06-30T13:14:16.589 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: cluster 2024-06-30T13:14:15.105473+0000 mgr.y (mgr.14144) 404 : cluster [DBG] pgmap v371: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:16.590 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[14758]: cluster 2024-06-30T13:14:15.288930+0000 mon.a (mon.0) 637 : cluster [WRN] Health check failed: Upgrade: failed to pull target image (UPGRADE_FAILED_PULL) 2024-06-30T13:14:16.590 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7912, in main 2024-06-30T13:14:16.590 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: r = ctx.func(ctx) 2024-06-30T13:14:16.590 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 1695, in _infer_image 2024-06-30T13:14:16.590 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: return func(ctx) 2024-06-30T13:14:16.590 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3196, in command_pull 2024-06-30T13:14:16.590 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: _pull_image(ctx, ctx.image) 2024-06-30T13:14:16.590 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3221, in _pull_image 2024-06-30T13:14:16.591 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: raise RuntimeError('Failed command: %s' % cmd_str) 2024-06-30T13:14:16.591 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: RuntimeError: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.591 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: cephadm 2024-06-30T13:14:15.098568+0000 mgr.y (mgr.14144) 403 : cephadm [ERR] Upgrade: Paused due to UPGRADE_FAILED_PULL: Upgrade: failed to pull target image 2024-06-30T13:14:16.591 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: cluster 2024-06-30T13:14:15.105473+0000 mgr.y (mgr.14144) 404 : cluster [DBG] pgmap v371: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:16.591 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:16 smithi144 bash[19242]: cluster 2024-06-30T13:14:15.288930+0000 mon.a (mon.0) 637 : cluster [WRN] Health check failed: Upgrade: failed to pull target image (UPGRADE_FAILED_PULL) 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: cluster 2024-06-30T13:14:14.523327+0000 mgr.y (mgr.14144) 400 : cluster [DBG] pgmap v370: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: cephadm 2024-06-30T13:14:14.565358+0000 mgr.y (mgr.14144) 401 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: cephadm 2024-06-30T13:14:15.098284+0000 mgr.y (mgr.14144) 402 : cephadm [ERR] cephadm exited with an error code: 1, stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65... 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: /usr/bin/docker: stderr Error response from daemon: unknown: Tag f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 was deleted or has expired. To pull, revive via time machine 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: Traceback (most recent call last): 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7924, in 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: main() 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7912, in main 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: r = ctx.func(ctx) 2024-06-30T13:14:16.779 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 1695, in _infer_image 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: return func(ctx) 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3196, in command_pull 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: _pull_image(ctx, ctx.image) 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3221, in _pull_image 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: raise RuntimeError('Failed command: %s' % cmd_str) 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: RuntimeError: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: Traceback (most recent call last): 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1172, in _remote_connection 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: yield (conn, connr) 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/usr/share/ceph/mgr/cephadm/serve.py", line 1087, in _run_cephadm 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: code, '\n'.join(err))) 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: orchestrator._interface.OrchestratorError: cephadm exited with an error code: 1, stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65... 2024-06-30T13:14:16.780 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: /usr/bin/docker: stderr Error response from daemon: unknown: Tag f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 was deleted or has expired. To pull, revive via time machine 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: Traceback (most recent call last): 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7924, in 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: main() 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7912, in main 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: r = ctx.func(ctx) 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 1695, in _infer_image 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: return func(ctx) 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3196, in command_pull 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: _pull_image(ctx, ctx.image) 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3221, in _pull_image 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: raise RuntimeError('Failed command: %s' % cmd_str) 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: RuntimeError: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: cephadm 2024-06-30T13:14:15.098568+0000 mgr.y (mgr.14144) 403 : cephadm [ERR] Upgrade: Paused due to UPGRADE_FAILED_PULL: Upgrade: failed to pull target image 2024-06-30T13:14:16.781 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: cluster 2024-06-30T13:14:15.105473+0000 mgr.y (mgr.14144) 404 : cluster [DBG] pgmap v371: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:16.782 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:16 smithi167 bash[14619]: cluster 2024-06-30T13:14:15.288930+0000 mon.a (mon.0) 637 : cluster [WRN] Health check failed: Upgrade: failed to pull target image (UPGRADE_FAILED_PULL) 2024-06-30T13:14:18.613 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:18 smithi144 bash[14758]: cluster 2024-06-30T13:14:17.106698+0000 mgr.y (mgr.14144) 405 : cluster [DBG] pgmap v372: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:18.613 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:18 smithi144 bash[19242]: cluster 2024-06-30T13:14:17.106698+0000 mgr.y (mgr.14144) 405 : cluster [DBG] pgmap v372: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:18.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:18 smithi167 bash[14619]: cluster 2024-06-30T13:14:17.106698+0000 mgr.y (mgr.14144) 405 : cluster [DBG] pgmap v372: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:19.041 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-06-30T13:14:19.389 INFO:teuthology.orchestra.run.smithi144.stdout:"Error: UPGRADE_FAILED_PULL: Upgrade: failed to pull target image" 2024-06-30T13:14:20.154 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph orch ps' 2024-06-30T13:14:20.456 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:20 smithi144 bash[19242]: audit 2024-06-30T13:14:19.024069+0000 mgr.y (mgr.14144) 406 : audit [DBG] from='client.24731 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:20.456 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:20 smithi144 bash[19242]: cluster 2024-06-30T13:14:19.107926+0000 mgr.y (mgr.14144) 407 : cluster [DBG] pgmap v373: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:20.456 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:20 smithi144 bash[19242]: audit 2024-06-30T13:14:19.372756+0000 mgr.y (mgr.14144) 408 : audit [DBG] from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:20.456 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:20 smithi144 bash[14758]: audit 2024-06-30T13:14:19.024069+0000 mgr.y (mgr.14144) 406 : audit [DBG] from='client.24731 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:20.456 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:20 smithi144 bash[14758]: cluster 2024-06-30T13:14:19.107926+0000 mgr.y (mgr.14144) 407 : cluster [DBG] pgmap v373: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:20.456 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:20 smithi144 bash[14758]: audit 2024-06-30T13:14:19.372756+0000 mgr.y (mgr.14144) 408 : audit [DBG] from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:20.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:20 smithi167 bash[14619]: audit 2024-06-30T13:14:19.024069+0000 mgr.y (mgr.14144) 406 : audit [DBG] from='client.24731 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:20.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:20 smithi167 bash[14619]: cluster 2024-06-30T13:14:19.107926+0000 mgr.y (mgr.14144) 407 : cluster [DBG] pgmap v373: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:20.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:20 smithi167 bash[14619]: audit 2024-06-30T13:14:19.372756+0000 mgr.y (mgr.14144) 408 : audit [DBG] from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:22.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:22 smithi144 bash[19242]: cluster 2024-06-30T13:14:21.109417+0000 mgr.y (mgr.14144) 409 : cluster [DBG] pgmap v374: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:22.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:22 smithi144 bash[14758]: cluster 2024-06-30T13:14:21.109417+0000 mgr.y (mgr.14144) 409 : cluster [DBG] pgmap v374: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:22.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:22 smithi167 bash[14619]: cluster 2024-06-30T13:14:21.109417+0000 mgr.y (mgr.14144) 409 : cluster [DBG] pgmap v374: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:23.811 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST STATUS REFRESHED AGE PORTS VERSION IMAGE ID CONTAINER ID 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.a smithi144 running (3m) 2m ago 4m *:9093 *:9094 0.20.0 0881eb8f169f 1e448f53e81c 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.a smithi167 running (4m) 2m ago 4m *:3000 6.7.4 557c83e11646 8fb29f5a6f9b 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:iscsi.foo.smithi167.teebbw smithi167 running (2m) 2m ago 2m - 3.4 24ecd6d5f14c f1a3417a4c13 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.x smithi167 running (9m) 2m ago 9m *:8443 16.2.0 24ecd6d5f14c 532cd7102966 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.y smithi144 running (11m) 2m ago 11m *:9283 16.2.0 24ecd6d5f14c 61d343ec135c 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:mon.a smithi144 running (11m) 2m ago 11m - 16.2.0 24ecd6d5f14c 19a0cb978436 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:mon.b smithi167 running (9m) 2m ago 9m - 16.2.0 24ecd6d5f14c 393f7e8aa7a0 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:mon.c smithi144 running (9m) 2m ago 9m - 16.2.0 24ecd6d5f14c f94b0072951c 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.a smithi144 running (4m) 2m ago 5m *:9100 0.18.1 e5a616e4b9cf 34292f3d8d6f 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.b smithi167 running (4m) 2m ago 5m *:9100 0.18.1 e5a616e4b9cf f2460a1db2c3 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (9m) 2m ago 9m - 16.2.0 24ecd6d5f14c b84b0a6c8235 2024-06-30T13:14:23.812 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (8m) 2m ago 8m - 16.2.0 24ecd6d5f14c e44f9e5f1582 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (8m) 2m ago 8m - 16.2.0 24ecd6d5f14c 3c7d64dd99ba 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi144 running (7m) 2m ago 7m - 16.2.0 24ecd6d5f14c 31d17c797c4e 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi167 running (7m) 2m ago 7m - 16.2.0 24ecd6d5f14c 479965722f27 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi167 running (6m) 2m ago 6m - 16.2.0 24ecd6d5f14c 45389120f4f0 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:osd.6 smithi167 running (6m) 2m ago 6m - 16.2.0 24ecd6d5f14c a43f1b565911 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:osd.7 smithi167 running (5m) 2m ago 5m - 16.2.0 24ecd6d5f14c 79b53c9abebe 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.a smithi167 running (3m) 2m ago 5m *:9095 2.18.1 de242295e225 e12f7c9952c3 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:rgw.foo.smithi144.pwxrka smithi144 running (2m) 2m ago 2m *:8000 16.2.0 24ecd6d5f14c 7e150ca0cd08 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:rgw.foo.smithi167.ojckkr smithi167 running (2m) 2m ago 2m *:8000 16.2.0 24ecd6d5f14c c7c210fa252d 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:rgw.smpl.smithi144.rfqrlh smithi144 running (2m) 2m ago 2m *:80 16.2.0 24ecd6d5f14c 2724fc438cf6 2024-06-30T13:14:23.813 INFO:teuthology.orchestra.run.smithi144.stdout:rgw.smpl.smithi167.onkqne smithi167 running (2m) 2m ago 2m *:80 16.2.0 24ecd6d5f14c 92410d41afbb 2024-06-30T13:14:24.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph versions' 2024-06-30T13:14:24.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:24 smithi144 bash[14758]: cluster 2024-06-30T13:14:23.110633+0000 mgr.y (mgr.14144) 410 : cluster [DBG] pgmap v375: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:24.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:24 smithi144 bash[19242]: cluster 2024-06-30T13:14:23.110633+0000 mgr.y (mgr.14144) 410 : cluster [DBG] pgmap v375: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:24.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:24 smithi167 bash[14619]: cluster 2024-06-30T13:14:23.110633+0000 mgr.y (mgr.14144) 410 : cluster [DBG] pgmap v375: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:25.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:25 smithi144 bash[19242]: audit 2024-06-30T13:14:23.794207+0000 mgr.y (mgr.14144) 411 : audit [DBG] from='client.24743 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:25.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:25 smithi144 bash[14758]: audit 2024-06-30T13:14:23.794207+0000 mgr.y (mgr.14144) 411 : audit [DBG] from='client.24743 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:25.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:25 smithi167 bash[14619]: audit 2024-06-30T13:14:23.794207+0000 mgr.y (mgr.14144) 411 : audit [DBG] from='client.24743 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:14:26.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:26 smithi144 bash[14758]: cluster 2024-06-30T13:14:25.112015+0000 mgr.y (mgr.14144) 412 : cluster [DBG] pgmap v376: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:26.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:26 smithi144 bash[19242]: cluster 2024-06-30T13:14:25.112015+0000 mgr.y (mgr.14144) 412 : cluster [DBG] pgmap v376: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:26.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:26 smithi167 bash[14619]: cluster 2024-06-30T13:14:25.112015+0000 mgr.y (mgr.14144) 412 : cluster [DBG] pgmap v376: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 967 B/s rd, 0 op/s 2024-06-30T13:14:28.356 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-06-30T13:14:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-06-30T13:14:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 3 2024-06-30T13:14:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:14:28.356 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 2 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 8 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": {}, 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "rgw": { 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 4 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 17 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:14:28.357 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-06-30T13:14:28.580 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:28 smithi144 bash[14758]: cluster 2024-06-30T13:14:27.113419+0000 mgr.y (mgr.14144) 413 : cluster [DBG] pgmap v377: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:28.580 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:28 smithi144 bash[19242]: cluster 2024-06-30T13:14:27.113419+0000 mgr.y (mgr.14144) 413 : cluster [DBG] pgmap v377: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:28.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:28 smithi167 bash[14619]: cluster 2024-06-30T13:14:27.113419+0000 mgr.y (mgr.14144) 413 : cluster [DBG] pgmap v377: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:29.108 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-06-30T13:14:29.362 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:29 smithi144 bash[19242]: audit 2024-06-30T13:14:28.352377+0000 mon.a (mon.0) 638 : audit [DBG] from='client.? 172.21.15.144:0/2567839106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:14:29.363 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:29 smithi144 bash[14758]: audit 2024-06-30T13:14:28.352377+0000 mon.a (mon.0) 638 : audit [DBG] from='client.? 172.21.15.144:0/2567839106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:14:29.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:29 smithi167 bash[14619]: audit 2024-06-30T13:14:28.352377+0000 mon.a (mon.0) 638 : audit [DBG] from='client.? 172.21.15.144:0/2567839106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:14:30.594 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:30 smithi144 bash[19242]: cluster 2024-06-30T13:14:29.114262+0000 mgr.y (mgr.14144) 414 : cluster [DBG] pgmap v378: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:30.594 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:30 smithi144 bash[14758]: cluster 2024-06-30T13:14:29.114262+0000 mgr.y (mgr.14144) 414 : cluster [DBG] pgmap v378: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:30.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:30 smithi167 bash[14619]: cluster 2024-06-30T13:14:29.114262+0000 mgr.y (mgr.14144) 414 : cluster [DBG] pgmap v378: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:32.417 INFO:teuthology.orchestra.run.smithi144.stdout:wait for servicemap items w/ changing names to refresh 2024-06-30T13:14:32.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:32 smithi144 bash[14758]: cluster 2024-06-30T13:14:31.115645+0000 mgr.y (mgr.14144) 415 : cluster [DBG] pgmap v379: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:32.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:32 smithi144 bash[19242]: cluster 2024-06-30T13:14:31.115645+0000 mgr.y (mgr.14144) 415 : cluster [DBG] pgmap v379: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:32.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:32 smithi167 bash[14619]: cluster 2024-06-30T13:14:31.115645+0000 mgr.y (mgr.14144) 415 : cluster [DBG] pgmap v379: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:33.291 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'sleep 60' 2024-06-30T13:14:34.616 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:34 smithi144 bash[14758]: cluster 2024-06-30T13:14:33.116871+0000 mgr.y (mgr.14144) 416 : cluster [DBG] pgmap v380: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:34.616 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:34 smithi144 bash[19242]: cluster 2024-06-30T13:14:33.116871+0000 mgr.y (mgr.14144) 416 : cluster [DBG] pgmap v380: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:34.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:34 smithi167 bash[14619]: cluster 2024-06-30T13:14:33.116871+0000 mgr.y (mgr.14144) 416 : cluster [DBG] pgmap v380: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:36.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:35 smithi144 bash[14758]: cluster 2024-06-30T13:14:35.118326+0000 mgr.y (mgr.14144) 417 : cluster [DBG] pgmap v381: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:36.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:35 smithi144 bash[19242]: cluster 2024-06-30T13:14:35.118326+0000 mgr.y (mgr.14144) 417 : cluster [DBG] pgmap v381: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:36.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:35 smithi167 bash[14619]: cluster 2024-06-30T13:14:35.118326+0000 mgr.y (mgr.14144) 417 : cluster [DBG] pgmap v381: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:38.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:38 smithi144 bash[19242]: cluster 2024-06-30T13:14:37.119684+0000 mgr.y (mgr.14144) 418 : cluster [DBG] pgmap v382: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:38.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:38 smithi144 bash[14758]: cluster 2024-06-30T13:14:37.119684+0000 mgr.y (mgr.14144) 418 : cluster [DBG] pgmap v382: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:38.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:38 smithi167 bash[14619]: cluster 2024-06-30T13:14:37.119684+0000 mgr.y (mgr.14144) 418 : cluster [DBG] pgmap v382: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:40.512 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:40 smithi144 bash[14758]: cluster 2024-06-30T13:14:39.120920+0000 mgr.y (mgr.14144) 419 : cluster [DBG] pgmap v383: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:40.512 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:40 smithi144 bash[19242]: cluster 2024-06-30T13:14:39.120920+0000 mgr.y (mgr.14144) 419 : cluster [DBG] pgmap v383: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:40.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:40 smithi167 bash[14619]: cluster 2024-06-30T13:14:39.120920+0000 mgr.y (mgr.14144) 419 : cluster [DBG] pgmap v383: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:42.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:42 smithi144 bash[19242]: cluster 2024-06-30T13:14:41.122491+0000 mgr.y (mgr.14144) 420 : cluster [DBG] pgmap v384: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:42.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:42 smithi144 bash[14758]: cluster 2024-06-30T13:14:41.122491+0000 mgr.y (mgr.14144) 420 : cluster [DBG] pgmap v384: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:42.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:42 smithi167 bash[14619]: cluster 2024-06-30T13:14:41.122491+0000 mgr.y (mgr.14144) 420 : cluster [DBG] pgmap v384: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:44.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:44 smithi144 bash[14758]: cluster 2024-06-30T13:14:43.123838+0000 mgr.y (mgr.14144) 421 : cluster [DBG] pgmap v385: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:44.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:44 smithi144 bash[19242]: cluster 2024-06-30T13:14:43.123838+0000 mgr.y (mgr.14144) 421 : cluster [DBG] pgmap v385: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:44.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:44 smithi167 bash[14619]: cluster 2024-06-30T13:14:43.123838+0000 mgr.y (mgr.14144) 421 : cluster [DBG] pgmap v385: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:46.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:45 smithi144 bash[14758]: cluster 2024-06-30T13:14:45.125290+0000 mgr.y (mgr.14144) 422 : cluster [DBG] pgmap v386: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:46.011 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:45 smithi144 bash[19242]: cluster 2024-06-30T13:14:45.125290+0000 mgr.y (mgr.14144) 422 : cluster [DBG] pgmap v386: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:46.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:45 smithi167 bash[14619]: cluster 2024-06-30T13:14:45.125290+0000 mgr.y (mgr.14144) 422 : cluster [DBG] pgmap v386: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:48.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:48 smithi144 bash[14758]: cluster 2024-06-30T13:14:47.126600+0000 mgr.y (mgr.14144) 423 : cluster [DBG] pgmap v387: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:48.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:48 smithi144 bash[19242]: cluster 2024-06-30T13:14:47.126600+0000 mgr.y (mgr.14144) 423 : cluster [DBG] pgmap v387: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:48.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:48 smithi167 bash[14619]: cluster 2024-06-30T13:14:47.126600+0000 mgr.y (mgr.14144) 423 : cluster [DBG] pgmap v387: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:50.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:50 smithi144 bash[14758]: cluster 2024-06-30T13:14:49.127840+0000 mgr.y (mgr.14144) 424 : cluster [DBG] pgmap v388: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:50.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:50 smithi144 bash[19242]: cluster 2024-06-30T13:14:49.127840+0000 mgr.y (mgr.14144) 424 : cluster [DBG] pgmap v388: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:50.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:50 smithi167 bash[14619]: cluster 2024-06-30T13:14:49.127840+0000 mgr.y (mgr.14144) 424 : cluster [DBG] pgmap v388: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:52.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:52 smithi144 bash[14758]: cluster 2024-06-30T13:14:51.129304+0000 mgr.y (mgr.14144) 425 : cluster [DBG] pgmap v389: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:52.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:52 smithi144 bash[19242]: cluster 2024-06-30T13:14:51.129304+0000 mgr.y (mgr.14144) 425 : cluster [DBG] pgmap v389: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:52.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:52 smithi167 bash[14619]: cluster 2024-06-30T13:14:51.129304+0000 mgr.y (mgr.14144) 425 : cluster [DBG] pgmap v389: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:54.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:54 smithi144 bash[14758]: cluster 2024-06-30T13:14:53.130427+0000 mgr.y (mgr.14144) 426 : cluster [DBG] pgmap v390: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:54.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:54 smithi144 bash[19242]: cluster 2024-06-30T13:14:53.130427+0000 mgr.y (mgr.14144) 426 : cluster [DBG] pgmap v390: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:54.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:54 smithi167 bash[14619]: cluster 2024-06-30T13:14:53.130427+0000 mgr.y (mgr.14144) 426 : cluster [DBG] pgmap v390: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:56.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:55 smithi167 bash[14619]: cluster 2024-06-30T13:14:55.131800+0000 mgr.y (mgr.14144) 427 : cluster [DBG] pgmap v391: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:56.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:55 smithi144 bash[14758]: cluster 2024-06-30T13:14:55.131800+0000 mgr.y (mgr.14144) 427 : cluster [DBG] pgmap v391: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:56.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:55 smithi144 bash[19242]: cluster 2024-06-30T13:14:55.131800+0000 mgr.y (mgr.14144) 427 : cluster [DBG] pgmap v391: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:14:57.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:56 smithi167 bash[14619]: audit 2024-06-30T13:14:56.559055+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:14:57.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:56 smithi167 bash[14619]: audit 2024-06-30T13:14:56.559552+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:14:57.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:56 smithi144 bash[14758]: audit 2024-06-30T13:14:56.559055+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:14:57.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:56 smithi144 bash[14758]: audit 2024-06-30T13:14:56.559552+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:14:57.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:56 smithi144 bash[19242]: audit 2024-06-30T13:14:56.559055+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:14:57.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:56 smithi144 bash[19242]: audit 2024-06-30T13:14:56.559552+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:14:58.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:14:57 smithi144 bash[14758]: cluster 2024-06-30T13:14:57.133166+0000 mgr.y (mgr.14144) 428 : cluster [DBG] pgmap v392: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:58.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:14:57 smithi144 bash[19242]: cluster 2024-06-30T13:14:57.133166+0000 mgr.y (mgr.14144) 428 : cluster [DBG] pgmap v392: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:14:58.277 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:14:57 smithi167 bash[14619]: cluster 2024-06-30T13:14:57.133166+0000 mgr.y (mgr.14144) 428 : cluster [DBG] pgmap v392: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:00.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:00 smithi144 bash[14758]: cluster 2024-06-30T13:14:59.134558+0000 mgr.y (mgr.14144) 429 : cluster [DBG] pgmap v393: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:00.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:00 smithi144 bash[19242]: cluster 2024-06-30T13:14:59.134558+0000 mgr.y (mgr.14144) 429 : cluster [DBG] pgmap v393: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:00.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:00 smithi167 bash[14619]: cluster 2024-06-30T13:14:59.134558+0000 mgr.y (mgr.14144) 429 : cluster [DBG] pgmap v393: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:02.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:02 smithi144 bash[14758]: cluster 2024-06-30T13:15:01.136163+0000 mgr.y (mgr.14144) 430 : cluster [DBG] pgmap v394: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:02.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:02 smithi144 bash[19242]: cluster 2024-06-30T13:15:01.136163+0000 mgr.y (mgr.14144) 430 : cluster [DBG] pgmap v394: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:02.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:02 smithi167 bash[14619]: cluster 2024-06-30T13:15:01.136163+0000 mgr.y (mgr.14144) 430 : cluster [DBG] pgmap v394: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:04 smithi144 bash[14758]: cluster 2024-06-30T13:15:03.137404+0000 mgr.y (mgr.14144) 431 : cluster [DBG] pgmap v395: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:04.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:04 smithi144 bash[19242]: cluster 2024-06-30T13:15:03.137404+0000 mgr.y (mgr.14144) 431 : cluster [DBG] pgmap v395: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:04.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:04 smithi167 bash[14619]: cluster 2024-06-30T13:15:03.137404+0000 mgr.y (mgr.14144) 431 : cluster [DBG] pgmap v395: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:06.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:05 smithi167 bash[14619]: cluster 2024-06-30T13:15:05.138737+0000 mgr.y (mgr.14144) 432 : cluster [DBG] pgmap v396: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:06.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:05 smithi144 bash[14758]: cluster 2024-06-30T13:15:05.138737+0000 mgr.y (mgr.14144) 432 : cluster [DBG] pgmap v396: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:06.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:05 smithi144 bash[19242]: cluster 2024-06-30T13:15:05.138737+0000 mgr.y (mgr.14144) 432 : cluster [DBG] pgmap v396: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:08.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:08 smithi144 bash[14758]: cluster 2024-06-30T13:15:07.140118+0000 mgr.y (mgr.14144) 433 : cluster [DBG] pgmap v397: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:08.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:08 smithi144 bash[19242]: cluster 2024-06-30T13:15:07.140118+0000 mgr.y (mgr.14144) 433 : cluster [DBG] pgmap v397: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:08.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:08 smithi167 bash[14619]: cluster 2024-06-30T13:15:07.140118+0000 mgr.y (mgr.14144) 433 : cluster [DBG] pgmap v397: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:10.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:10 smithi144 bash[14758]: cluster 2024-06-30T13:15:09.141538+0000 mgr.y (mgr.14144) 434 : cluster [DBG] pgmap v398: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:10.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:10 smithi144 bash[19242]: cluster 2024-06-30T13:15:09.141538+0000 mgr.y (mgr.14144) 434 : cluster [DBG] pgmap v398: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:10.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:10 smithi167 bash[14619]: cluster 2024-06-30T13:15:09.141538+0000 mgr.y (mgr.14144) 434 : cluster [DBG] pgmap v398: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:12.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:12 smithi144 bash[19242]: cluster 2024-06-30T13:15:11.143090+0000 mgr.y (mgr.14144) 435 : cluster [DBG] pgmap v399: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:12.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:12 smithi144 bash[14758]: cluster 2024-06-30T13:15:11.143090+0000 mgr.y (mgr.14144) 435 : cluster [DBG] pgmap v399: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:12.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:12 smithi167 bash[14619]: cluster 2024-06-30T13:15:11.143090+0000 mgr.y (mgr.14144) 435 : cluster [DBG] pgmap v399: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:14.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:14 smithi144 bash[14758]: cluster 2024-06-30T13:15:13.144279+0000 mgr.y (mgr.14144) 436 : cluster [DBG] pgmap v400: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:14.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:14 smithi144 bash[19242]: cluster 2024-06-30T13:15:13.144279+0000 mgr.y (mgr.14144) 436 : cluster [DBG] pgmap v400: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:14.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:14 smithi167 bash[14619]: cluster 2024-06-30T13:15:13.144279+0000 mgr.y (mgr.14144) 436 : cluster [DBG] pgmap v400: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:16.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:15 smithi167 bash[14619]: audit 2024-06-30T13:15:15.122240+0000 mon.a (mon.0) 641 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:15:16.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:15 smithi167 bash[14619]: cluster 2024-06-30T13:15:15.145674+0000 mgr.y (mgr.14144) 437 : cluster [DBG] pgmap v401: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:16.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:15 smithi167 bash[14619]: audit 2024-06-30T13:15:15.412019+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:15:16.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:15 smithi144 bash[14758]: audit 2024-06-30T13:15:15.122240+0000 mon.a (mon.0) 641 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:15:16.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:15 smithi144 bash[14758]: cluster 2024-06-30T13:15:15.145674+0000 mgr.y (mgr.14144) 437 : cluster [DBG] pgmap v401: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:16.261 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:15 smithi144 bash[14758]: audit 2024-06-30T13:15:15.412019+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:15:16.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:15 smithi144 bash[19242]: audit 2024-06-30T13:15:15.122240+0000 mon.a (mon.0) 641 : audit [DBG] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-30T13:15:16.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:15 smithi144 bash[19242]: cluster 2024-06-30T13:15:15.145674+0000 mgr.y (mgr.14144) 437 : cluster [DBG] pgmap v401: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:16.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:15 smithi144 bash[19242]: audit 2024-06-30T13:15:15.412019+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' 2024-06-30T13:15:18.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:18 smithi144 bash[14758]: cluster 2024-06-30T13:15:17.147075+0000 mgr.y (mgr.14144) 438 : cluster [DBG] pgmap v402: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:18.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:18 smithi144 bash[19242]: cluster 2024-06-30T13:15:17.147075+0000 mgr.y (mgr.14144) 438 : cluster [DBG] pgmap v402: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:18.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:18 smithi167 bash[14619]: cluster 2024-06-30T13:15:17.147075+0000 mgr.y (mgr.14144) 438 : cluster [DBG] pgmap v402: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:20.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:20 smithi144 bash[14758]: cluster 2024-06-30T13:15:19.148349+0000 mgr.y (mgr.14144) 439 : cluster [DBG] pgmap v403: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:20.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:20 smithi144 bash[19242]: cluster 2024-06-30T13:15:19.148349+0000 mgr.y (mgr.14144) 439 : cluster [DBG] pgmap v403: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:20.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:20 smithi167 bash[14619]: cluster 2024-06-30T13:15:19.148349+0000 mgr.y (mgr.14144) 439 : cluster [DBG] pgmap v403: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:22.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:22 smithi144 bash[14758]: cluster 2024-06-30T13:15:21.149788+0000 mgr.y (mgr.14144) 440 : cluster [DBG] pgmap v404: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:22.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:22 smithi144 bash[19242]: cluster 2024-06-30T13:15:21.149788+0000 mgr.y (mgr.14144) 440 : cluster [DBG] pgmap v404: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:22.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:22 smithi167 bash[14619]: cluster 2024-06-30T13:15:21.149788+0000 mgr.y (mgr.14144) 440 : cluster [DBG] pgmap v404: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:24.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:24 smithi144 bash[14758]: cluster 2024-06-30T13:15:23.151042+0000 mgr.y (mgr.14144) 441 : cluster [DBG] pgmap v405: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:24.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:24 smithi144 bash[19242]: cluster 2024-06-30T13:15:23.151042+0000 mgr.y (mgr.14144) 441 : cluster [DBG] pgmap v405: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:24.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:24 smithi167 bash[14619]: cluster 2024-06-30T13:15:23.151042+0000 mgr.y (mgr.14144) 441 : cluster [DBG] pgmap v405: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:26.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:25 smithi167 bash[14619]: cluster 2024-06-30T13:15:25.152427+0000 mgr.y (mgr.14144) 442 : cluster [DBG] pgmap v406: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:26.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:25 smithi144 bash[14758]: cluster 2024-06-30T13:15:25.152427+0000 mgr.y (mgr.14144) 442 : cluster [DBG] pgmap v406: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:26.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:25 smithi144 bash[19242]: cluster 2024-06-30T13:15:25.152427+0000 mgr.y (mgr.14144) 442 : cluster [DBG] pgmap v406: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:28.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:28 smithi144 bash[14758]: cluster 2024-06-30T13:15:27.153903+0000 mgr.y (mgr.14144) 443 : cluster [DBG] pgmap v407: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:28.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:28 smithi144 bash[19242]: cluster 2024-06-30T13:15:27.153903+0000 mgr.y (mgr.14144) 443 : cluster [DBG] pgmap v407: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:28.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:28 smithi167 bash[14619]: cluster 2024-06-30T13:15:27.153903+0000 mgr.y (mgr.14144) 443 : cluster [DBG] pgmap v407: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:30.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:30 smithi144 bash[14758]: cluster 2024-06-30T13:15:29.155203+0000 mgr.y (mgr.14144) 444 : cluster [DBG] pgmap v408: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:30.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:30 smithi144 bash[19242]: cluster 2024-06-30T13:15:29.155203+0000 mgr.y (mgr.14144) 444 : cluster [DBG] pgmap v408: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:30.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:30 smithi167 bash[14619]: cluster 2024-06-30T13:15:29.155203+0000 mgr.y (mgr.14144) 444 : cluster [DBG] pgmap v408: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:32.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:32 smithi144 bash[14758]: cluster 2024-06-30T13:15:31.156624+0000 mgr.y (mgr.14144) 445 : cluster [DBG] pgmap v409: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:32.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:32 smithi144 bash[19242]: cluster 2024-06-30T13:15:31.156624+0000 mgr.y (mgr.14144) 445 : cluster [DBG] pgmap v409: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:32.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:32 smithi167 bash[14619]: cluster 2024-06-30T13:15:31.156624+0000 mgr.y (mgr.14144) 445 : cluster [DBG] pgmap v409: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:34.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:34 smithi144 bash[14758]: cluster 2024-06-30T13:15:33.157828+0000 mgr.y (mgr.14144) 446 : cluster [DBG] pgmap v410: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:34.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:34 smithi144 bash[19242]: cluster 2024-06-30T13:15:33.157828+0000 mgr.y (mgr.14144) 446 : cluster [DBG] pgmap v410: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:34.527 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:34 smithi167 bash[14619]: cluster 2024-06-30T13:15:33.157828+0000 mgr.y (mgr.14144) 446 : cluster [DBG] pgmap v410: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:36.028 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:35 smithi167 bash[14619]: cluster 2024-06-30T13:15:35.159270+0000 mgr.y (mgr.14144) 447 : cluster [DBG] pgmap v411: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:36.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:35 smithi144 bash[14758]: cluster 2024-06-30T13:15:35.159270+0000 mgr.y (mgr.14144) 447 : cluster [DBG] pgmap v411: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:36.260 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:35 smithi144 bash[19242]: cluster 2024-06-30T13:15:35.159270+0000 mgr.y (mgr.14144) 447 : cluster [DBG] pgmap v411: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:37.552 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph orch ps' 2024-06-30T13:15:38.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:38 smithi144 bash[19242]: cluster 2024-06-30T13:15:37.160656+0000 mgr.y (mgr.14144) 448 : cluster [DBG] pgmap v412: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:38.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:38 smithi144 bash[14758]: cluster 2024-06-30T13:15:37.160656+0000 mgr.y (mgr.14144) 448 : cluster [DBG] pgmap v412: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:38.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:38 smithi167 bash[14619]: cluster 2024-06-30T13:15:37.160656+0000 mgr.y (mgr.14144) 448 : cluster [DBG] pgmap v412: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:40.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:40 smithi144 bash[19242]: cluster 2024-06-30T13:15:39.161995+0000 mgr.y (mgr.14144) 449 : cluster [DBG] pgmap v413: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:40.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:40 smithi144 bash[14758]: cluster 2024-06-30T13:15:39.161995+0000 mgr.y (mgr.14144) 449 : cluster [DBG] pgmap v413: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:40.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:40 smithi167 bash[14619]: cluster 2024-06-30T13:15:39.161995+0000 mgr.y (mgr.14144) 449 : cluster [DBG] pgmap v413: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:41.436 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST STATUS REFRESHED AGE PORTS VERSION IMAGE ID CONTAINER ID 2024-06-30T13:15:41.436 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.a smithi144 running (5m) 3m ago 6m *:9093 *:9094 0.20.0 0881eb8f169f 1e448f53e81c 2024-06-30T13:15:41.436 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.a smithi167 running (5m) 3m ago 5m *:3000 6.7.4 557c83e11646 8fb29f5a6f9b 2024-06-30T13:15:41.436 INFO:teuthology.orchestra.run.smithi144.stdout:iscsi.foo.smithi167.teebbw smithi167 running (3m) 3m ago 3m - 3.4 24ecd6d5f14c f1a3417a4c13 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.x smithi167 running (10m) 3m ago 10m *:8443 16.2.0 24ecd6d5f14c 532cd7102966 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.y smithi144 running (12m) 3m ago 12m *:9283 16.2.0 24ecd6d5f14c 61d343ec135c 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:mon.a smithi144 running (12m) 3m ago 12m - 16.2.0 24ecd6d5f14c 19a0cb978436 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:mon.b smithi167 running (11m) 3m ago 11m - 16.2.0 24ecd6d5f14c 393f7e8aa7a0 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:mon.c smithi144 running (11m) 3m ago 11m - 16.2.0 24ecd6d5f14c f94b0072951c 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.a smithi144 running (6m) 3m ago 6m *:9100 0.18.1 e5a616e4b9cf 34292f3d8d6f 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.b smithi167 running (6m) 3m ago 6m *:9100 0.18.1 e5a616e4b9cf f2460a1db2c3 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (10m) 3m ago 10m - 16.2.0 24ecd6d5f14c b84b0a6c8235 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (9m) 3m ago 9m - 16.2.0 24ecd6d5f14c e44f9e5f1582 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (9m) 3m ago 9m - 16.2.0 24ecd6d5f14c 3c7d64dd99ba 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi144 running (8m) 3m ago 8m - 16.2.0 24ecd6d5f14c 31d17c797c4e 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi167 running (8m) 3m ago 8m - 16.2.0 24ecd6d5f14c 479965722f27 2024-06-30T13:15:41.437 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi167 running (8m) 3m ago 8m - 16.2.0 24ecd6d5f14c 45389120f4f0 2024-06-30T13:15:41.438 INFO:teuthology.orchestra.run.smithi144.stdout:osd.6 smithi167 running (7m) 3m ago 7m - 16.2.0 24ecd6d5f14c a43f1b565911 2024-06-30T13:15:41.438 INFO:teuthology.orchestra.run.smithi144.stdout:osd.7 smithi167 running (7m) 3m ago 7m - 16.2.0 24ecd6d5f14c 79b53c9abebe 2024-06-30T13:15:41.438 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.a smithi167 running (5m) 3m ago 6m *:9095 2.18.1 de242295e225 e12f7c9952c3 2024-06-30T13:15:41.438 INFO:teuthology.orchestra.run.smithi144.stdout:rgw.foo.smithi144.pwxrka smithi144 running (4m) 3m ago 4m *:8000 16.2.0 24ecd6d5f14c 7e150ca0cd08 2024-06-30T13:15:41.438 INFO:teuthology.orchestra.run.smithi144.stdout:rgw.foo.smithi167.ojckkr smithi167 running (4m) 3m ago 4m *:8000 16.2.0 24ecd6d5f14c c7c210fa252d 2024-06-30T13:15:41.438 INFO:teuthology.orchestra.run.smithi144.stdout:rgw.smpl.smithi144.rfqrlh smithi144 running (3m) 3m ago 3m *:80 16.2.0 24ecd6d5f14c 2724fc438cf6 2024-06-30T13:15:41.438 INFO:teuthology.orchestra.run.smithi144.stdout:rgw.smpl.smithi167.onkqne smithi167 running (3m) 3m ago 3m *:80 16.2.0 24ecd6d5f14c 92410d41afbb 2024-06-30T13:15:42.196 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph versions' 2024-06-30T13:15:42.447 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:42 smithi144 bash[14758]: cluster 2024-06-30T13:15:41.163409+0000 mgr.y (mgr.14144) 450 : cluster [DBG] pgmap v414: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:42.448 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:42 smithi144 bash[14758]: audit 2024-06-30T13:15:41.411854+0000 mgr.y (mgr.14144) 451 : audit [DBG] from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:15:42.448 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:42 smithi144 bash[19242]: cluster 2024-06-30T13:15:41.163409+0000 mgr.y (mgr.14144) 450 : cluster [DBG] pgmap v414: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:42.448 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:42 smithi144 bash[19242]: audit 2024-06-30T13:15:41.411854+0000 mgr.y (mgr.14144) 451 : audit [DBG] from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:15:42.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:42 smithi167 bash[14619]: cluster 2024-06-30T13:15:41.163409+0000 mgr.y (mgr.14144) 450 : cluster [DBG] pgmap v414: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:42.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:42 smithi167 bash[14619]: audit 2024-06-30T13:15:41.411854+0000 mgr.y (mgr.14144) 451 : audit [DBG] from='client.14832 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:15:44.511 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:44 smithi144 bash[14758]: cluster 2024-06-30T13:15:43.164596+0000 mgr.y (mgr.14144) 452 : cluster [DBG] pgmap v415: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:44.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:44 smithi144 bash[19242]: cluster 2024-06-30T13:15:43.164596+0000 mgr.y (mgr.14144) 452 : cluster [DBG] pgmap v415: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:44.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:44 smithi167 bash[14619]: cluster 2024-06-30T13:15:43.164596+0000 mgr.y (mgr.14144) 452 : cluster [DBG] pgmap v415: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:46.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:45 smithi144 bash[14758]: cluster 2024-06-30T13:15:45.165995+0000 mgr.y (mgr.14144) 453 : cluster [DBG] pgmap v416: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:46.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:45 smithi144 bash[19242]: cluster 2024-06-30T13:15:45.165995+0000 mgr.y (mgr.14144) 453 : cluster [DBG] pgmap v416: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 3 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 2 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 8 2024-06-30T13:15:46.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:15:46.029 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": {}, 2024-06-30T13:15:46.029 INFO:teuthology.orchestra.run.smithi144.stdout: "rgw": { 2024-06-30T13:15:46.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 4 2024-06-30T13:15:46.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-06-30T13:15:46.029 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-06-30T13:15:46.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable)": 17 2024-06-30T13:15:46.029 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-06-30T13:15:46.029 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-06-30T13:15:46.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:45 smithi167 bash[14619]: cluster 2024-06-30T13:15:45.165995+0000 mgr.y (mgr.14144) 453 : cluster [DBG] pgmap v416: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:46.791 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph orch upgrade status' 2024-06-30T13:15:47.049 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:46 smithi144 bash[19242]: audit 2024-06-30T13:15:46.024726+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.144:0/2577506236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:15:47.050 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:46 smithi144 bash[14758]: audit 2024-06-30T13:15:46.024726+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.144:0/2577506236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:15:47.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:46 smithi167 bash[14619]: audit 2024-06-30T13:15:46.024726+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? 172.21.15.144:0/2577506236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:15:48.095 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:47 smithi144 bash[14758]: cluster 2024-06-30T13:15:47.167336+0000 mgr.y (mgr.14144) 454 : cluster [DBG] pgmap v417: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:48.096 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:47 smithi144 bash[19242]: cluster 2024-06-30T13:15:47.167336+0000 mgr.y (mgr.14144) 454 : cluster [DBG] pgmap v417: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:48.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:47 smithi167 bash[14619]: cluster 2024-06-30T13:15:47.167336+0000 mgr.y (mgr.14144) 454 : cluster [DBG] pgmap v417: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:50.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:50 smithi144 bash[14758]: cluster 2024-06-30T13:15:49.168480+0000 mgr.y (mgr.14144) 455 : cluster [DBG] pgmap v418: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:50.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:50 smithi144 bash[19242]: cluster 2024-06-30T13:15:49.168480+0000 mgr.y (mgr.14144) 455 : cluster [DBG] pgmap v418: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:50.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:50 smithi167 bash[14619]: cluster 2024-06-30T13:15:49.168480+0000 mgr.y (mgr.14144) 455 : cluster [DBG] pgmap v418: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:15:50.626 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-06-30T13:15:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65", 2024-06-30T13:15:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-06-30T13:15:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [], 2024-06-30T13:15:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "", 2024-06-30T13:15:50.627 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Error: UPGRADE_FAILED_PULL: Upgrade: failed to pull target image" 2024-06-30T13:15:50.628 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-06-30T13:15:51.393 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph health detail' 2024-06-30T13:15:52.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:52 smithi144 bash[14758]: audit 2024-06-30T13:15:50.622856+0000 mgr.y (mgr.14144) 456 : audit [DBG] from='client.24764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:15:52.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:52 smithi144 bash[14758]: cluster 2024-06-30T13:15:51.170037+0000 mgr.y (mgr.14144) 457 : cluster [DBG] pgmap v419: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:52.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:52 smithi144 bash[19242]: audit 2024-06-30T13:15:50.622856+0000 mgr.y (mgr.14144) 456 : audit [DBG] from='client.24764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:15:52.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:52 smithi144 bash[19242]: cluster 2024-06-30T13:15:51.170037+0000 mgr.y (mgr.14144) 457 : cluster [DBG] pgmap v419: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:52.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:52 smithi167 bash[14619]: audit 2024-06-30T13:15:50.622856+0000 mgr.y (mgr.14144) 456 : audit [DBG] from='client.24764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-30T13:15:52.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:52 smithi167 bash[14619]: cluster 2024-06-30T13:15:51.170037+0000 mgr.y (mgr.14144) 457 : cluster [DBG] pgmap v419: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:54.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:54 smithi144 bash[14758]: cluster 2024-06-30T13:15:53.171290+0000 mgr.y (mgr.14144) 458 : cluster [DBG] pgmap v420: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-30T13:15:54.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:54 smithi144 bash[19242]: cluster 2024-06-30T13:15:53.171290+0000 mgr.y (mgr.14144) 458 : cluster [DBG] pgmap v420: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-30T13:15:54.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:54 smithi167 bash[14619]: cluster 2024-06-30T13:15:53.171290+0000 mgr.y (mgr.14144) 458 : cluster [DBG] pgmap v420: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-30T13:15:55.153 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN Upgrade: failed to pull target image 2024-06-30T13:15:55.153 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] UPGRADE_FAILED_PULL: Upgrade: failed to pull target image 2024-06-30T13:15:55.153 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi144 `cephadm pull` failed: cephadm exited with an error code: 1, stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65... 2024-06-30T13:15:55.153 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:15:55.153 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/docker: stderr Error response from daemon: unknown: Tag f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 was deleted or has expired. To pull, revive via time machine 2024-06-30T13:15:55.153 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-06-30T13:15:55.153 INFO:teuthology.orchestra.run.smithi144.stdout: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7924, in 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout: main() 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 7912, in main 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout: r = ctx.func(ctx) 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 1695, in _infer_image 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout: return func(ctx) 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3196, in command_pull 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout: _pull_image(ctx, ctx.image) 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout: File "/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/cephadm.f77d9d71514a634758d4ad41ab6eef36d25386c99d8b365310ad41f9b74d5ce6", line 3221, in _pull_image 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout: raise RuntimeError('Failed command: %s' % cmd_str) 2024-06-30T13:15:55.154 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 2024-06-30T13:15:55.960 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-06-30T13:15:56.219 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:55 smithi144 bash[19242]: audit 2024-06-30T13:15:55.149828+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.144:0/3310456491' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-30T13:15:56.219 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:55 smithi144 bash[19242]: cluster 2024-06-30T13:15:55.172663+0000 mgr.y (mgr.14144) 459 : cluster [DBG] pgmap v421: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-30T13:15:56.219 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:55 smithi144 bash[14758]: audit 2024-06-30T13:15:55.149828+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.144:0/3310456491' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-30T13:15:56.219 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:55 smithi144 bash[14758]: cluster 2024-06-30T13:15:55.172663+0000 mgr.y (mgr.14144) 459 : cluster [DBG] pgmap v421: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-30T13:15:56.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:55 smithi167 bash[14619]: audit 2024-06-30T13:15:55.149828+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? 172.21.15.144:0/3310456491' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-06-30T13:15:56.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:55 smithi167 bash[14619]: cluster 2024-06-30T13:15:55.172663+0000 mgr.y (mgr.14144) 459 : cluster [DBG] pgmap v421: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-30T13:15:57.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:56 smithi144 bash[14758]: audit 2024-06-30T13:15:56.571702+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:15:57.260 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:56 smithi144 bash[14758]: audit 2024-06-30T13:15:56.572341+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:15:57.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:56 smithi144 bash[19242]: audit 2024-06-30T13:15:56.571702+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:15:57.261 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:56 smithi144 bash[19242]: audit 2024-06-30T13:15:56.572341+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:15:57.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:56 smithi167 bash[14619]: audit 2024-06-30T13:15:56.571702+0000 mon.a (mon.0) 643 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-30T13:15:57.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:56 smithi167 bash[14619]: audit 2024-06-30T13:15:56.572341+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.14144 172.21.15.144:0/3038997666' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-30T13:15:58.010 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:15:57 smithi144 bash[14758]: cluster 2024-06-30T13:15:57.174343+0000 mgr.y (mgr.14144) 460 : cluster [DBG] pgmap v422: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:58.010 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:15:57 smithi144 bash[19242]: cluster 2024-06-30T13:15:57.174343+0000 mgr.y (mgr.14144) 460 : cluster [DBG] pgmap v422: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:58.278 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:15:57 smithi167 bash[14619]: cluster 2024-06-30T13:15:57.174343+0000 mgr.y (mgr.14144) 460 : cluster [DBG] pgmap v422: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:15:59.821 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-06-30T13:16:00.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:16:00 smithi144 bash[14758]: cluster 2024-06-30T13:15:59.175682+0000 mgr.y (mgr.14144) 461 : cluster [DBG] pgmap v423: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-30T13:16:00.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:16:00 smithi144 bash[14758]: audit 2024-06-30T13:15:59.799967+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? 172.21.15.144:0/3951080076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:16:00.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:16:00 smithi144 bash[19242]: cluster 2024-06-30T13:15:59.175682+0000 mgr.y (mgr.14144) 461 : cluster [DBG] pgmap v423: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-30T13:16:00.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:16:00 smithi144 bash[19242]: audit 2024-06-30T13:15:59.799967+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? 172.21.15.144:0/3951080076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:16:00.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:16:00 smithi167 bash[14619]: cluster 2024-06-30T13:15:59.175682+0000 mgr.y (mgr.14144) 461 : cluster [DBG] pgmap v423: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-06-30T13:16:00.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:16:00 smithi167 bash[14619]: audit 2024-06-30T13:15:59.799967+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? 172.21.15.144:0/3951080076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:16:00.549 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-06-30T13:16:02.510 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:16:02 smithi144 bash[19242]: cluster 2024-06-30T13:16:01.177328+0000 mgr.y (mgr.14144) 462 : cluster [DBG] pgmap v424: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:16:02.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:16:02 smithi144 bash[14758]: cluster 2024-06-30T13:16:01.177328+0000 mgr.y (mgr.14144) 462 : cluster [DBG] pgmap v424: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:16:02.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:16:02 smithi167 bash[14619]: cluster 2024-06-30T13:16:01.177328+0000 mgr.y (mgr.14144) 462 : cluster [DBG] pgmap v424: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-06-30T13:16:04.510 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:16:04 smithi144 bash[14758]: cluster 2024-06-30T13:16:03.178669+0000 mgr.y (mgr.14144) 463 : cluster [DBG] pgmap v425: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:16:04.511 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:16:04 smithi144 bash[19242]: cluster 2024-06-30T13:16:03.178669+0000 mgr.y (mgr.14144) 463 : cluster [DBG] pgmap v425: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:16:04.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:16:04 smithi167 bash[14619]: cluster 2024-06-30T13:16:03.178669+0000 mgr.y (mgr.14144) 463 : cluster [DBG] pgmap v425: 161 pgs: 161 active+clean; 27 KiB data, 88 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-06-30T13:16:05.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-30T13:16:05.173 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_16b16b98599950a27899283412b0a32cd03699ac/qa/tasks/cephadm.py", line 1307, in shell _shell(ctx, cluster_name, remote, File "/home/teuthworker/src/git.ceph.com_ceph_16b16b98599950a27899283412b0a32cd03699ac/qa/tasks/cephadm.py", line 39, in _shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi144 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | keys\'"\'"\' | grep $sha1\'' 2024-06-30T13:16:05.346 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a663d683834b448a8136754ed1b4cffc Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_16b16b98599950a27899283412b0a32cd03699ac/qa/tasks/cephadm.py", line 1307, in shell _shell(ctx, cluster_name, remote, File "/home/teuthworker/src/git.ceph.com_ceph_16b16b98599950a27899283412b0a32cd03699ac/qa/tasks/cephadm.py", line 39, in _shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi144 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | keys\'"\'"\' | grep $sha1\'' 2024-06-30T13:16:05.349 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-30T13:16:05.358 INFO:tasks.cephadm:Teardown begin 2024-06-30T13:16:05.358 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-30T13:16:05.378 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-30T13:16:05.395 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-30T13:16:05.395 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-30T13:16:05.425 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-30T13:16:05.431 INFO:tasks.cephadm:Stopping all daemons... 2024-06-30T13:16:05.432 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-30T13:16:05.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a 2024-06-30T13:16:05.444 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:16:05 smithi144 bash[14758]: audit 2024-06-30T13:16:04.405835+0000 mon.a (mon.0) 645 : audit [DBG] from='client.? 172.21.15.144:0/3479390595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:16:05.444 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:16:05 smithi144 bash[19242]: audit 2024-06-30T13:16:04.405835+0000 mon.a (mon.0) 645 : audit [DBG] from='client.? 172.21.15.144:0/3479390595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:16:05.528 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:16:05 smithi167 bash[14619]: audit 2024-06-30T13:16:04.405835+0000 mon.a (mon.0) 645 : audit [DBG] from='client.? 172.21.15.144:0/3479390595' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-06-30T13:16:05.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:16:05 smithi144 systemd[1]: Stopping Ceph mon.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:05.760 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:16:05 smithi144 bash[14758]: debug 2024-06-30T13:16:05.553+0000 7fa6b1383700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-06-30T13:16:05.761 INFO:journalctl@ceph.mon.a.smithi144.stdout:Jun 30 13:16:05 smithi144 bash[14758]: debug 2024-06-30T13:16:05.553+0000 7fa6b1383700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-30T13:16:06.378 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.a.service' 2024-06-30T13:16:06.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:06.420 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-30T13:16:06.421 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-06-30T13:16:06.421 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.c 2024-06-30T13:16:06.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:16:06 smithi144 systemd[1]: Stopping Ceph mon.c for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:06.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:16:06 smithi144 bash[19242]: debug 2024-06-30T13:16:06.593+0000 7f3d539ee700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-06-30T13:16:06.760 INFO:journalctl@ceph.mon.c.smithi144.stdout:Jun 30 13:16:06 smithi144 bash[19242]: debug 2024-06-30T13:16:06.593+0000 7f3d539ee700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-06-30T13:16:07.394 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.c.service' 2024-06-30T13:16:07.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:07.430 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-06-30T13:16:07.430 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-06-30T13:16:07.430 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.b 2024-06-30T13:16:07.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:16:07 smithi167 systemd[1]: Stopping Ceph mon.b for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:07.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:16:07 smithi167 bash[14619]: debug 2024-06-30T13:16:07.531+0000 7f0edb371700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-06-30T13:16:07.778 INFO:journalctl@ceph.mon.b.smithi167.stdout:Jun 30 13:16:07 smithi167 bash[14619]: debug 2024-06-30T13:16:07.531+0000 7f0edb371700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-06-30T13:16:08.534 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mon.b.service' 2024-06-30T13:16:08.581 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:08.581 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-06-30T13:16:08.582 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-30T13:16:08.582 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y 2024-06-30T13:16:08.929 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:16:08 smithi144 systemd[1]: Stopping Ceph mgr.y for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:08.929 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:16:08 smithi144 bash[15024]: debug 2024-06-30T13:16:08.685+0000 7f01a8ba7700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-06-30T13:16:08.930 INFO:journalctl@ceph.mgr.y.smithi144.stdout:Jun 30 13:16:08 smithi144 bash[15024]: debug 2024-06-30T13:16:08.685+0000 7f01a8ba7700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-30T13:16:09.456 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.y.service' 2024-06-30T13:16:09.492 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:09.492 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-30T13:16:09.493 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-30T13:16:09.493 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.x 2024-06-30T13:16:09.778 INFO:journalctl@ceph.mgr.x.smithi167.stdout:Jun 30 13:16:09 smithi167 systemd[1]: Stopping Ceph mgr.x for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:10.279 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@mgr.x.service' 2024-06-30T13:16:10.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:10.311 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-30T13:16:10.311 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-30T13:16:10.312 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.0 2024-06-30T13:16:10.670 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:16:10 smithi144 systemd[1]: Stopping Ceph osd.0 for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:10.670 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:16:10 smithi144 bash[21982]: debug 2024-06-30T13:16:10.429+0000 7f6ff2fb5700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-06-30T13:16:10.670 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:16:10 smithi144 bash[21982]: debug 2024-06-30T13:16:10.429+0000 7f6ff2fb5700 -1 osd.0 71 *** Got signal Terminated *** 2024-06-30T13:16:10.670 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:16:10 smithi144 bash[21982]: debug 2024-06-30T13:16:10.429+0000 7f6ff2fb5700 -1 osd.0 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-30T13:16:11.378 INFO:journalctl@ceph.osd.0.smithi144.stdout:Jun 30 13:16:11 smithi144 docker[45040]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-osd.0 2024-06-30T13:16:13.321 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.0.service' 2024-06-30T13:16:13.356 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:13.356 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-30T13:16:13.356 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-30T13:16:13.356 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.1 2024-06-30T13:16:13.695 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:16:13 smithi144 systemd[1]: Stopping Ceph osd.1 for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:13.695 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:16:13 smithi144 bash[24683]: debug 2024-06-30T13:16:13.457+0000 7f9d1e77c700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-06-30T13:16:13.695 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:16:13 smithi144 bash[24683]: debug 2024-06-30T13:16:13.457+0000 7f9d1e77c700 -1 osd.1 71 *** Got signal Terminated *** 2024-06-30T13:16:13.695 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:16:13 smithi144 bash[24683]: debug 2024-06-30T13:16:13.457+0000 7f9d1e77c700 -1 osd.1 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-30T13:16:14.312 INFO:journalctl@ceph.osd.1.smithi144.stdout:Jun 30 13:16:14 smithi144 docker[45174]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-osd.1 2024-06-30T13:16:16.253 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.1.service' 2024-06-30T13:16:16.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:16.288 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-30T13:16:16.288 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-30T13:16:16.289 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.2 2024-06-30T13:16:16.510 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:16:16 smithi144 systemd[1]: Stopping Ceph osd.2 for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:16.511 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:16:16 smithi144 bash[27411]: debug 2024-06-30T13:16:16.453+0000 7faff3315700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-06-30T13:16:16.511 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:16:16 smithi144 bash[27411]: debug 2024-06-30T13:16:16.453+0000 7faff3315700 -1 osd.2 71 *** Got signal Terminated *** 2024-06-30T13:16:16.511 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:16:16 smithi144 bash[27411]: debug 2024-06-30T13:16:16.453+0000 7faff3315700 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-30T13:16:17.362 INFO:journalctl@ceph.osd.2.smithi144.stdout:Jun 30 13:16:17 smithi144 docker[45319]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-osd.2 2024-06-30T13:16:19.329 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.2.service' 2024-06-30T13:16:19.362 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:19.362 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-30T13:16:19.362 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-30T13:16:19.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.3 2024-06-30T13:16:19.694 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:16:19 smithi144 systemd[1]: Stopping Ceph osd.3 for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:19.694 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:16:19 smithi144 bash[29945]: debug 2024-06-30T13:16:19.461+0000 7f89f8c75700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-06-30T13:16:19.695 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:16:19 smithi144 bash[29945]: debug 2024-06-30T13:16:19.461+0000 7f89f8c75700 -1 osd.3 71 *** Got signal Terminated *** 2024-06-30T13:16:19.695 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:16:19 smithi144 bash[29945]: debug 2024-06-30T13:16:19.461+0000 7f89f8c75700 -1 osd.3 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-30T13:16:20.334 INFO:journalctl@ceph.osd.3.smithi144.stdout:Jun 30 13:16:20 smithi144 docker[45453]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-osd.3 2024-06-30T13:16:22.264 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.3.service' 2024-06-30T13:16:22.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:22.301 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-30T13:16:22.301 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-30T13:16:22.301 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.4 2024-06-30T13:16:22.663 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:16:22 smithi167 systemd[1]: Stopping Ceph osd.4 for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:22.664 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:16:22 smithi167 bash[17918]: debug 2024-06-30T13:16:22.423+0000 7fa105741700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-06-30T13:16:22.664 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:16:22 smithi167 bash[17918]: debug 2024-06-30T13:16:22.423+0000 7fa105741700 -1 osd.4 71 *** Got signal Terminated *** 2024-06-30T13:16:22.664 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:16:22 smithi167 bash[17918]: debug 2024-06-30T13:16:22.423+0000 7fa105741700 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-30T13:16:23.251 INFO:journalctl@ceph.osd.4.smithi167.stdout:Jun 30 13:16:22 smithi167 docker[33290]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-osd.4 2024-06-30T13:16:25.191 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.4.service' 2024-06-30T13:16:25.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:25.221 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-30T13:16:25.221 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-30T13:16:25.221 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.5 2024-06-30T13:16:25.528 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:16:25 smithi167 systemd[1]: Stopping Ceph osd.5 for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:25.528 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:16:25 smithi167 bash[20455]: debug 2024-06-30T13:16:25.307+0000 7f940d287700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-06-30T13:16:25.528 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:16:25 smithi167 bash[20455]: debug 2024-06-30T13:16:25.307+0000 7f940d287700 -1 osd.5 71 *** Got signal Terminated *** 2024-06-30T13:16:25.529 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:16:25 smithi167 bash[20455]: debug 2024-06-30T13:16:25.307+0000 7f940d287700 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-30T13:16:26.114 INFO:journalctl@ceph.osd.5.smithi167.stdout:Jun 30 13:16:25 smithi167 docker[33435]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-osd.5 2024-06-30T13:16:28.043 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.5.service' 2024-06-30T13:16:28.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:28.075 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-30T13:16:28.076 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-30T13:16:28.076 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.6 2024-06-30T13:16:28.413 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:16:28 smithi167 systemd[1]: Stopping Ceph osd.6 for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:28.413 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:16:28 smithi167 bash[23355]: debug 2024-06-30T13:16:28.171+0000 7fed1526b700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-06-30T13:16:28.413 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:16:28 smithi167 bash[23355]: debug 2024-06-30T13:16:28.171+0000 7fed1526b700 -1 osd.6 71 *** Got signal Terminated *** 2024-06-30T13:16:28.414 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:16:28 smithi167 bash[23355]: debug 2024-06-30T13:16:28.171+0000 7fed1526b700 -1 osd.6 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-30T13:16:28.968 INFO:journalctl@ceph.osd.6.smithi167.stdout:Jun 30 13:16:28 smithi167 docker[33572]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-osd.6 2024-06-30T13:16:30.806 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.6.service' 2024-06-30T13:16:30.835 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:30.835 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-30T13:16:30.836 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-30T13:16:30.836 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.7 2024-06-30T13:16:31.179 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:16:30 smithi167 systemd[1]: Stopping Ceph osd.7 for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:31.179 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:16:30 smithi167 bash[25867]: debug 2024-06-30T13:16:30.931+0000 7f56b50e2700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-06-30T13:16:31.179 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:16:30 smithi167 bash[25867]: debug 2024-06-30T13:16:30.931+0000 7f56b50e2700 -1 osd.7 71 *** Got signal Terminated *** 2024-06-30T13:16:31.179 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:16:30 smithi167 bash[25867]: debug 2024-06-30T13:16:30.931+0000 7f56b50e2700 -1 osd.7 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-30T13:16:31.733 INFO:journalctl@ceph.osd.7.smithi167.stdout:Jun 30 13:16:31 smithi167 docker[33703]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-osd.7 2024-06-30T13:16:33.590 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@osd.7.service' 2024-06-30T13:16:33.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:33.623 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-30T13:16:33.623 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-06-30T13:16:33.623 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@prometheus.a 2024-06-30T13:16:33.920 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 systemd[1]: Stopping Ceph prometheus.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:33.921 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=warn ts=2024-06-30T13:16:33.712Z caller=main.go:524 msg="Received SIGTERM, exiting gracefully..." 2024-06-30T13:16:33.921 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.712Z caller=main.go:547 msg="Stopping scrape discovery manager..." 2024-06-30T13:16:33.921 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.712Z caller=main.go:561 msg="Stopping notify discovery manager..." 2024-06-30T13:16:33.921 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.712Z caller=main.go:583 msg="Stopping scrape manager..." 2024-06-30T13:16:33.921 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.712Z caller=main.go:557 msg="Notify discovery manager stopped" 2024-06-30T13:16:33.921 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.712Z caller=main.go:543 msg="Scrape discovery manager stopped" 2024-06-30T13:16:33.922 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.713Z caller=manager.go:882 component="rule manager" msg="Stopping rule manager..." 2024-06-30T13:16:33.922 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.713Z caller=manager.go:892 component="rule manager" msg="Rule manager stopped" 2024-06-30T13:16:33.922 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.713Z caller=main.go:577 msg="Scrape manager stopped" 2024-06-30T13:16:33.922 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.713Z caller=notifier.go:601 component=notifier msg="Stopping notification manager..." 2024-06-30T13:16:33.922 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.713Z caller=main.go:749 msg="Notifier manager stopped" 2024-06-30T13:16:33.922 INFO:journalctl@ceph.prometheus.a.smithi167.stdout:Jun 30 13:16:33 smithi167 bash[29292]: level=info ts=2024-06-30T13:16:33.714Z caller=main.go:761 msg="See you next time!" 2024-06-30T13:16:34.425 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@prometheus.a.service' 2024-06-30T13:16:34.458 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-30T13:16:34.458 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-06-30T13:16:34.458 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 --force --keep-logs 2024-06-30T13:16:34.614 INFO:teuthology.orchestra.run.smithi144.stderr:usage: cephadm [-h] [--image IMAGE] [--docker] [--data-dir DATA_DIR] 2024-06-30T13:16:34.615 INFO:teuthology.orchestra.run.smithi144.stderr: [--log-dir LOG_DIR] [--logrotate-dir LOGROTATE_DIR] 2024-06-30T13:16:34.615 INFO:teuthology.orchestra.run.smithi144.stderr: [--unit-dir UNIT_DIR] [--verbose] [--timeout TIMEOUT] 2024-06-30T13:16:34.615 INFO:teuthology.orchestra.run.smithi144.stderr: [--retry RETRY] [--env ENV] [--no-container-init] 2024-06-30T13:16:34.615 INFO:teuthology.orchestra.run.smithi144.stderr: {version,pull,inspect-image,ls,list-networks,adopt,rm-daemon,rm-cluster,run,shell,enter,ceph-volume,unit,logs,bootstrap,deploy,check-host,prepare-host,add-repo,rm-repo,install,registry-login,gather-facts,exporter,host-maintenance,verify-prereqs} 2024-06-30T13:16:34.615 INFO:teuthology.orchestra.run.smithi144.stderr: ... 2024-06-30T13:16:34.615 INFO:teuthology.orchestra.run.smithi144.stderr:cephadm: error: unrecognized arguments: --keep-logs 2024-06-30T13:16:34.630 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-06-30T13:16:34.631 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 --force --keep-logs 2024-06-30T13:16:34.791 INFO:teuthology.orchestra.run.smithi167.stderr:usage: cephadm [-h] [--image IMAGE] [--docker] [--data-dir DATA_DIR] 2024-06-30T13:16:34.791 INFO:teuthology.orchestra.run.smithi167.stderr: [--log-dir LOG_DIR] [--logrotate-dir LOGROTATE_DIR] 2024-06-30T13:16:34.791 INFO:teuthology.orchestra.run.smithi167.stderr: [--unit-dir UNIT_DIR] [--verbose] [--timeout TIMEOUT] 2024-06-30T13:16:34.792 INFO:teuthology.orchestra.run.smithi167.stderr: [--retry RETRY] [--env ENV] [--no-container-init] 2024-06-30T13:16:34.792 INFO:teuthology.orchestra.run.smithi167.stderr: {version,pull,inspect-image,ls,list-networks,adopt,rm-daemon,rm-cluster,run,shell,enter,ceph-volume,unit,logs,bootstrap,deploy,check-host,prepare-host,add-repo,rm-repo,install,registry-login,gather-facts,exporter,host-maintenance,verify-prereqs} 2024-06-30T13:16:34.792 INFO:teuthology.orchestra.run.smithi167.stderr: ... 2024-06-30T13:16:34.792 INFO:teuthology.orchestra.run.smithi167.stderr:cephadm: error: unrecognized arguments: --keep-logs 2024-06-30T13:16:34.806 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-06-30T13:16:34.806 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-30T13:16:34.826 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-30T13:16:34.837 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-30T13:16:34.839 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592/remote/smithi144/crash 2024-06-30T13:16:34.840 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/crash -- . 2024-06-30T13:16:34.885 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592/remote/smithi167/crash 2024-06-30T13:16:34.886 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/crash -- . 2024-06-30T13:16:34.904 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-30T13:16:34.904 DEBUG:teuthology.orchestra.run.smithi144:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-06-30T13:16:34.919 INFO:teuthology.orchestra.run.smithi144.stderr:grep: /var/log/ceph/f6ab1df6-36e0-11ef-bcaa-c7b262605968/ceph.log: No such file or directory 2024-06-30T13:16:34.921 INFO:tasks.cephadm:Compressing logs... 2024-06-30T13:16:34.921 DEBUG:teuthology.orchestra.run.smithi144:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-30T13:16:34.965 DEBUG:teuthology.orchestra.run.smithi167:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-30T13:16:34.978 INFO:teuthology.orchestra.run.smithi144.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-06-30T13:16:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-06-30T13:16:37.322 INFO:tasks.cephadm:Archiving logs... 2024-06-30T13:16:37.322 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592/remote/smithi144/log 2024-06-30T13:16:37.324 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-30T13:16:37.654 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592/remote/smithi167/log 2024-06-30T13:16:37.655 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-30T13:16:37.895 INFO:tasks.cephadm:Removing cluster... 2024-06-30T13:16:37.895 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 --force 2024-06-30T13:16:42.358 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:16:42 smithi144 systemd[1]: Stopping Ceph node-exporter.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:43.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:16:42 smithi144 docker[45814]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-node-exporter.a 2024-06-30T13:16:43.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:16:43 smithi144 systemd[1]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-06-30T13:16:43.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:16:43 smithi144 systemd[1]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-06-30T13:16:43.260 INFO:journalctl@ceph.node-exporter.a.smithi144.stdout:Jun 30 13:16:43 smithi144 systemd[1]: Stopped Ceph node-exporter.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:16:43.606 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:16:43 smithi144 systemd[1]: Stopping Ceph alertmanager.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:16:43.606 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:16:43 smithi144 bash[36617]: level=info ts=2024-06-30T13:16:43.417Z caller=main.go:536 msg="Received SIGTERM, exiting gracefully..." 2024-06-30T13:16:43.923 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:16:43 smithi144 docker[45886]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-alertmanager.a 2024-06-30T13:16:44.260 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:16:44 smithi144 systemd[1]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@alertmanager.a.service: Succeeded. 2024-06-30T13:16:44.260 INFO:journalctl@ceph.alertmanager.a.smithi144.stdout:Jun 30 13:16:44 smithi144 systemd[1]: Stopped Ceph alertmanager.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:17:06.762 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 --force 2024-06-30T13:17:11.228 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:17:11 smithi167 systemd[1]: Stopping Ceph node-exporter.b for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:17:12.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:17:11 smithi167 docker[34105]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-node-exporter.b 2024-06-30T13:17:12.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:17:11 smithi167 systemd[1]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-06-30T13:17:12.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:17:11 smithi167 systemd[1]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-06-30T13:17:12.028 INFO:journalctl@ceph.node-exporter.b.smithi167.stdout:Jun 30 13:17:11 smithi167 systemd[1]: Stopped Ceph node-exporter.b for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:17:12.432 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 systemd[1]: Stopping Ceph grafana.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968... 2024-06-30T13:17:12.432 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-06-30T13:17:12.432 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped RemoteCache" logger=server reason="context canceled" 2024-06-30T13:17:12.432 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped NotificationService" logger=server reason="context canceled" 2024-06-30T13:17:12.432 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped InternalMetricsService" logger=server reason="context canceled" 2024-06-30T13:17:12.432 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped provisioningServiceImpl" logger=server reason="context canceled" 2024-06-30T13:17:12.433 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped PluginManager" logger=server reason="context canceled" 2024-06-30T13:17:12.433 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped BackendPluginManager" logger=server reason="context canceled" 2024-06-30T13:17:12.433 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped CleanUpService" logger=server reason="context canceled" 2024-06-30T13:17:12.433 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped UserAuthTokenService" logger=server reason="context canceled" 2024-06-30T13:17:12.433 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped AlertEngine" logger=server reason="context canceled" 2024-06-30T13:17:12.433 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped Stream Manager" 2024-06-30T13:17:12.433 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 bash[28664]: t=2024-06-30T13:17:12+0000 lvl=info msg="Stopped UsageStatsService" logger=server reason="context canceled" 2024-06-30T13:17:12.998 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 docker[34180]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968-grafana.a 2024-06-30T13:17:13.278 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 systemd[1]: ceph-f6ab1df6-36e0-11ef-bcaa-c7b262605968@grafana.a.service: Succeeded. 2024-06-30T13:17:13.278 INFO:journalctl@ceph.grafana.a.smithi167.stdout:Jun 30 13:17:12 smithi167 systemd[1]: Stopped Ceph grafana.a for f6ab1df6-36e0-11ef-bcaa-c7b262605968. 2024-06-30T13:17:47.212 INFO:tasks.cephadm:Removing cephadm ... 2024-06-30T13:17:47.213 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-30T13:17:47.220 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-30T13:17:47.225 INFO:tasks.cephadm:Teardown complete 2024-06-30T13:17:47.226 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-30T13:17:47.234 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-30T13:17:47.235 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-30T13:17:47.237 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-30T13:17:47.257 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2024-06-30T13:17:47.258 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2024-06-30T13:17:47.258 INFO:teuthology.orchestra.run.smithi144.stdout:+hv01.front.sepi 67.205.162.81 3 u 15 64 377 0.083 1.211 0.404 2024-06-30T13:17:47.258 INFO:teuthology.orchestra.run.smithi144.stdout:+hv02.front.sepi 63.231.80.2 3 u 11 64 377 0.088 3.221 0.787 2024-06-30T13:17:47.259 INFO:teuthology.orchestra.run.smithi144.stdout:*hv03.front.sepi 74.6.168.72 3 u 19 64 377 0.079 3.660 0.441 2024-06-30T13:17:47.259 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 0.000 0.000 2024-06-30T13:17:47.290 INFO:teuthology.orchestra.run.smithi167.stdout: remote refid st t when poll reach delay offset jitter 2024-06-30T13:17:47.290 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================== 2024-06-30T13:17:47.290 INFO:teuthology.orchestra.run.smithi167.stdout:+hv01.front.sepi 67.205.162.81 3 u 23 64 377 0.088 -0.567 0.870 2024-06-30T13:17:47.291 INFO:teuthology.orchestra.run.smithi167.stdout:+hv02.front.sepi 63.231.80.2 3 u 13 64 377 0.100 2.457 0.590 2024-06-30T13:17:47.291 INFO:teuthology.orchestra.run.smithi167.stdout:*hv03.front.sepi 74.6.168.72 3 u 19 64 377 0.083 1.720 0.819 2024-06-30T13:17:47.291 INFO:teuthology.orchestra.run.smithi167.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 0.000 0.000 2024-06-30T13:17:47.292 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-30T13:17:47.300 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-30T13:17:47.301 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-30T13:17:47.323 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-30T13:17:47.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-30T13:17:47.390 INFO:teuthology.task.internal:Duration was 1270.351812 seconds 2024-06-30T13:17:47.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-30T13:17:47.424 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-30T13:17:47.424 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-30T13:17:47.427 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-30T13:17:47.479 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-30T13:17:47.479 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-06-30T13:17:47.480 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-30T13:17:47.493 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-06-30T13:17:47.493 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-30T13:17:47.508 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-30T13:17:47.508 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-30T13:17:47.510 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-30T13:17:47.604 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-30T13:17:47.604 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-30T13:17:47.606 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-30T13:17:47.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-30T13:17:47.729 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-30T13:17:47.729 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-30T13:17:47.742 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-30T13:17:47.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-30T13:17:47.769 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-30T13:17:47.772 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-30T13:17:47.783 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-06-30T13:17:47.814 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-06-30T13:17:47.839 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-30T13:17:47.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-30T13:17:47.860 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-30T13:17:47.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-30T13:17:47.866 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-30T13:17:47.876 INFO:teuthology.task.internal:Transferring archived files... 2024-06-30T13:17:47.877 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592/remote/smithi144 2024-06-30T13:17:47.877 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-30T13:17:47.908 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-10_22:08:02-orch-reef-distro-default-smithi/7749592/remote/smithi167 2024-06-30T13:17:47.908 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-30T13:17:47.935 INFO:teuthology.task.internal:Removing archive directory... 2024-06-30T13:17:47.935 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-30T13:17:47.944 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-30T13:17:47.978 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-30T13:17:47.990 INFO:teuthology.task.internal:Not uploading archives. 2024-06-30T13:17:47.990 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-30T13:17:48.008 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-30T13:17:48.008 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-30T13:17:48.010 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-30T13:17:48.015 INFO:teuthology.orchestra.run.smithi144.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 30 13:17 /home/ubuntu/cephtest 2024-06-30T13:17:48.018 INFO:teuthology.orchestra.run.smithi167.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 30 13:17 /home/ubuntu/cephtest 2024-06-30T13:17:48.020 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-30T13:17:48.042 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-30T13:17:48.111 INFO:teuthology.run:Summary data: description: orch/cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} duration: 1270.3518116474152 failure_reason: 'Command failed on smithi144 with status 1: ''sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v16.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f6ab1df6-36e0-11ef-bcaa-c7b262605968 -e sha1=f1abf795f4fc15bf2f30ac90c811a53ab21d5d65 -- bash -c \''ceph versions | jq -e \''"\''"\''.overall | keys\''"\''"\'' | grep $sha1\''''' owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a663d683834b448a8136754ed1b4cffc status: fail success: false 2024-06-30T13:17:48.112 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-30T13:17:48.193 INFO:teuthology.run:FAIL